Archive

Posts Tagged ‘python3’

differences between Python 2 and Python 3 with examples

January 7, 2016 Leave a comment

http://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html

Advertisements
Categories: python Tags: ,

creating a Python 3 virt. env. on Ubuntu

Problem
Python 3 on Ubuntu is sometimes a total mess. I wanted to create a Python 3 virt. env., but I got this error:

$ virtualenv -p python3 venv
Running virtualenv with interpreter /usr/bin/python3
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 8, in 
    import base64
  File "/usr/lib/python3.4/base64.py", line 9, in 
    import re
  File "/usr/lib/python3.4/re.py", line 324, in 
    import copyreg
  File "/usr/local/lib/python2.7/dist-packages/copyreg.py", line 3, in 
    from copy_reg import *
ImportError: No module named 'copy_reg'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 53, in apport_excepthook
    if not enabled():
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 24, in enabled
    import re
  File "/usr/lib/python3.4/re.py", line 324, in 
    import copyreg
  File "/usr/local/lib/python2.7/dist-packages/copyreg.py", line 3, in 
    from copy_reg import *
ImportError: No module named 'copy_reg'

Original exception was:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 8, in 
    import base64
  File "/usr/lib/python3.4/base64.py", line 9, in 
    import re
  File "/usr/lib/python3.4/re.py", line 324, in 
    import copyreg
  File "/usr/local/lib/python2.7/dist-packages/copyreg.py", line 3, in 
    from copy_reg import *
ImportError: No module named 'copy_reg'

Awesome! :(

Solution
I found a working solution here. The following command let me create a Python 3 virt. env.:

python3 -c 'import sys; del sys.argv[0]; s = sys.argv[0]; exec(open(s).read(), {"__file__": s, "__name__": "__main__"})' `which virtualenv` -p python3 venv 

Life on Arch / Manjaro is easier. There Python 3 has been the default for years.

pyvenv: create virtual environmets for Python 3.4+

January 1, 2015 Leave a comment

For creating virtual environmets, I’ve used virtualenvwrapper so far. However, Python 3.4 contains the command pyvenv that does the same thing. Since it also installs pip in the virt. env., it can replace virtualenvwrapper.

I like to store my virtual environments in a dedicated folder, separated from the project directory. virtualenvwrapper, by default, stores the virt. env.’s in the ~/.virtualenvs folder. Since I got used to this folder, I will continue to keep my virt. env.’s in this folder.

pyvenv
Say we have our project folder here: ~/python/webapps/flasky_project. Create a virt. env. for this the following way:

pyvenv ~/.virtualenvs/flasky_project

It will create a Python 3 virt. env.

virtualenv / virtualenvwrapper
For the sake of completeness, I also write here how to create virt. env.’s with virtualenv and virtualenvwrapper:

# blog post: http://goo.gl/oEdtT3

# virtualenvwrapper for Python 3 or Python 2
mkvirtualenv -p `which python3` myenv3
mkvirtualenv -p `which python2` myenv2

# virtualenv for Python 3 or Python 2
virtualenv -p python3 myproject3
virtualenv -p python2 myproject2

# When the env. is created, activate it
# and launch the command python within.
# Verify if it's the correct version.

TL; DR
If you use Python 3.4+ and you need a virt. env., use the command “pyvenv“.

XML to dict / XML to JSON

December 29, 2014 Leave a comment

Problem
You have an XML file and you want to convert it to dict or JSON.

Well, if you have a dict, you can convert it to JSON with “json.dump()“, so the real question is: how to convert an XML file to a dictionary?

Solution
There is an excellent library for this purpose called xmltodict. Its usage is very simple:

import xmltodict

# It doesn't work with Python 3! Read on for the solution!
def convert(xml_file, xml_attribs=True):
    with open(xml_file) as f:
        d = xmltodict.parse(f, xml_attribs=xml_attribs)
        return d

This worked well under Python 2.7 but I got an error under Python 3. I checked the project’s documentation and it claimed to be Python 3 compatible. What the hell?

The error message was this:

Traceback (most recent call last):
  File "/home/jabba/Dropbox/python/lib/jabbapylib2/apps/xmltodict.py", line 247, in parse
    parser.ParseFile(xml_input)
TypeError: read() did not return a bytes object (type=str)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./xml2json.py", line 27, in <module>
    print(convert(sys.argv[1]))
  File "./xml2json.py", line 17, in convert
    d = xmltodict.parse(f, xml_attribs=xml_attribs)
  File "/home/jabba/Dropbox/python/lib/jabbapylib2/apps/xmltodict.py", line 249, in parse
    parser.Parse(xml_input, True)
TypeError: '_io.TextIOWrapper' does not support the buffer interface

I even filed an issue ticket :)

After some debugging I found a hint here: you need to open the XML file in binary mode!

XML to dict (Python 2 & 3)
So the correct version that works with Python 3 too is this:

import xmltodict

def convert(xml_file, xml_attribs=True):
    with open(xml_file, "rb") as f:    # notice the "rb" mode
        d = xmltodict.parse(f, xml_attribs=xml_attribs)
        return d

XML to JSON (Python 2 & 3)
If you want JSON output:

import json
import xmltodict

def convert(xml_file, xml_attribs=True):
    with open(xml_file, "rb") as f:    # notice the "rb" mode
        d = xmltodict.parse(f, xml_attribs=xml_attribs)
        return json.dumps(d, indent=4)
Categories: python Tags: , , , , , ,

make a script run under Python 2.x and 3.x too

November 3, 2014 Leave a comment

Problem
I installed Manjaro Linux on one of my laptops, just to try something new. I’ve been using it for a week and I like it so far :) On my older laptop it runs smoother than Ubuntu.

Anyway, Manjaro switched to Python 3.x, that’s the default, thus “python” points to Python 3. I use Ubuntu on my other machines where Python 2 is the default. I would like to modify my scripts (at least some of them) to run on both systems.

For instance, in Python 2.x you call “raw_input”, while this function was renamed to “input” in Python 3.x.

Solution
Well, since January 2014 I start all my new scripts with this line:

from __future__ import (absolute_import, division,
                        print_function, unicode_literals)

It ensures a nice transition from Python 2 to Python 3.

To solve the “raw_input” problem, you can add these lines:

import sys

if sys.version_info >= (3, 0):
    raw_input = input

You can continue using “raw_input”, but if it’s executed with Python 3.x, “raw_input” will point to the “input” function.

Of course, the ideal solution would be to switch to Python 3, but I’m not ready for that yet :)

Update (20141228)
At the moment I’m updating my jabbapylib library. The new version will be released soon :) Since it’s a library, it should work with both Python 2 and Python 3. When I write a new script, I tend to use Python 3 these days, but a library is different. A library should support both Python 2.x and 3.x. The most widely used solution is the Six compatibility library, which is a joy to use. To solve the raw_input issue for instance, just import the line

from six.moves import input

Then — just like in Python 3 — call the function “input()” to read from the standard input. For more info. read the official docs.

Categories: python Tags: , , ,

make bpython use Python 3

October 14, 2014 1 comment

Problem
On my Linux box the default Python is version 2.7. I installed bpython system-wide, thus when I launch bpython, it uses Python 2.7.

How could I have another bpython that uses Python 3? I would like to have both though. Sometimes I need Python 2, sometimes Python 3, and I would like to use the bpython shell with them.

Solution
Simply install bpython in a Python 3 virtual environment. That is, create a Python 3 environment, and install bpython in it with pip. This bpython installation will use Python 3. Then use the absolute path of this bpython executable. And no, you don’t have to activate the virt. env. each time you want to use this bpython.

In a previous post I showed how to patch virtualenvwrapper to make it suitable for creating not only Python 2 but Python 3 environments too. Using this patched virtualenvwrapper, the process looked like this on my machine:

$ cd
$ mkdir bpython_python3
$ cd bpython_python3
$ mkvirtualenv bpython_python3  # creates a Python 3 virt. env.
(bpython_python3)$ pip install bpython
(bpython_python3)$ which bpython
/home/jabba/.virtualenvs/bpython_python3/bin/bpython  # sample; will be different in your case
(bpython_python3)$ deactivate
$

I suggest adding the following aliases to your $HOME/.bashrc:

alias bpy='bpython'
alias bpy3='/home/jabba/.virtualenvs/bpython_python3/bin/bpython'

Of course, you need to customize the path for bpy3.

Now “bpy” launches bpython with Python 2, while “bpy3” starts bpython with Python 3.

Categories: ubuntu Tags: ,

[django] virtualenvwrapper with Python 3

September 12, 2014 Leave a comment

Problem
Django 1.7 came out ten days ago. If you look at the updated poll tutorial, you will notice that it’s written for Python 3! So far I’ve used Python 2 only, so I thought I would redo this tutorial for two reasons: (1) learn about migrations, and (2) start using Python 3.

For my Django projects I use virtualenvwrapper to manage my virtual environments, so the first question was: how to create a Python 3 virt. env. with virtualenvwrapper?

Well, maybe my installation got corrupted somehow, but I ran into this problem:

$ mkvirtualenv -p `which python3` tmpenv
Running virtualenv with interpreter /usr/bin/python3
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 8, in <module>
    import base64
  File "/usr/lib/python3.4/base64.py", line 9, in <module>
    import re
  File "/usr/lib/python3.4/re.py", line 324, in <module>
    import copyreg
  File "/usr/local/lib/python2.7/dist-packages/copyreg.py", line 3, in <module>
    from copy_reg import *
ImportError: No module named 'copy_reg'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 53, in apport_excepthook
    if not enabled():
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 24, in enabled
    import re
  File "/usr/lib/python3.4/re.py", line 324, in <module>
    import copyreg
  File "/usr/local/lib/python2.7/dist-packages/copyreg.py", line 3, in <module>
    from copy_reg import *
ImportError: No module named 'copy_reg'

Original exception was:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 8, in <module>
    import base64
  File "/usr/lib/python3.4/base64.py", line 9, in <module>
    import re
  File "/usr/lib/python3.4/re.py", line 324, in <module>
    import copyreg
  File "/usr/local/lib/python2.7/dist-packages/copyreg.py", line 3, in <module>
    from copy_reg import *
ImportError: No module named 'copy_reg'

Solution
I posted this question to reddit (see here) and I found the following solution:

First, install pip3:

sudo apt-get install python3-pip
sudo pip3 install virtualenvwrapper

Then, add the following lines to your .bashrc:

export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/workspace    # customize if needed
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv
source /usr/local/bin/virtualenvwrapper.sh

Try if it works for you. In my case, I had to edit the file “/usr/local/bin/virtualenv“. I changed its first line to

#!/usr/bin/env python3

The consequence of changing the first line was that creating Python 3 environments became the default, but at least it works now.

Create a Python 3 virt. env.:

mktmpenv

Create a Python 2 virt. env.:

mktmpenv -p `which python2`

If you use the command “virtualenv“, it will also create Python 3 environments by default. Thus, “virtualenv myproject3” will be a Python 3 env., while “virtualenv -p python2 myproject2” will be a Python 2 env.