Posts Tagged ‘project layout’

[django] virtualenvwrapper + optimal project layout + requirements.txt

August 4, 2014 Leave a comment

Here we will see how to use virtualenvwrapper, how to create a requirements.txt file, and what is the optimal layout for a Django project.

Here I suppose you already know virtualenv. virtualenvwrapper, as its name suggests, is a wrapper for virtualenv, making your life even easier (especially when you have several projects).

First, install it globally:

sudo pip install virtualenvwrapper -U

I keep my Django projects in a dedicated folder, which is “$HOME/Dropbox/python/webapps” in my case. I put them to Dropbox, thus (1) I have a simple backup solution, and (2) I can work on a project on any machine of mine since the Dropbox client synchronizes everything. The path of the folder above is too long, so I have a symbolic link “$HOME/webapps” that points to “$HOME/Dropbox/python/webapps“.

Now, add the following lines to the end of your .bashrc file:

# virtualenvwrapper
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/webapps
source /usr/local/bin/

Open a new terminal to activate the new settings.

The good thing is that virtual environments will be separated from our project directory. That is, instead of “$HOME/webapps“, virtual environments will be kept in the “$HOME/.virtualenvs” folder. A virtual env. can be quite large, but it won’t fill your Dropbox space since it’s kept outside.

At this time you might ask: “OK, my Django project is in Dropbox, but how can I work on it on two different machines if the virtual env. is outside of Dropbox?” That’s a good question. This is why we will use a “requirements.txt” file, which collects all the packages that are installed in the virtual environment. This file will be in the project folder (i.e., inside Dropbox). If you want to work on a project on a new machine, using the “requirements.txt” file you can re-create the virtual env., which will be identical to the one on your first machine. We will see it later in the “requirements.txt” section.

project layout
The 3rd chapter of the book “Two Scoops of Django” suggests a three-tiered approach. I like it, so let’s see it through an example. Let’s create a new Django project called “hello“. Enter the folder “$HOME/webapps” and create the project directory (it is the 1st tier). I always add the “_project” suffix to a project (later we will see a bash trick that relies on it, so let’s call the folder “hello_project“). We are here at the moment:

jabba@jabba-uplink:~$ cd
jabba@jabba-uplink:~$ cd webapps
jabba@jabba-uplink:~/webapps$ mkdir hello_project
jabba@jabba-uplink:~/webapps$ cd hello_project/
jabba@jabba-uplink:~/webapps/hello_project$ ls -al
total 8
drwxrwxr-x 2 jabba jabba 4096 Aug  4 16:04 .
drwxrwxr-x 7 jabba jabba 4096 Aug  4 16:04 ..

Now let’s create a virtual env. for the project:

jabba@jabba-uplink:~/webapps/hello_project$ mkvirtualenv hello_project
New python executable in hello_project/bin/python
Installing setuptools, pip...done.
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ ls -al
total 8
drwxrwxr-x 2 jabba jabba 4096 Aug  4 16:04 .
drwxrwxr-x 7 jabba jabba 4096 Aug  4 16:04 ..

We use the “mkvirtualenv” command. The name of the virtual env. can be the same as the name of the project (actually, in our case they should be called the same — we will see later why). This way you can associate them easily. Notice two things: (1) the prompt has changed, indicating that the “hello_project” virtual env. has been activated (“mkvirtualenv” creates a virtual env. and also activates it), and (2) the current folder is still empty, i.e. the virtual env. was created somewhere else.

Where is the virtual env.? Go to “$HOME/.virtualenvs” and there you will see a subfolder “hello_project” that contains the virtual environment of our new Django project.

Extra stuff: the basic bash prompt is monochrome and thus not too appealing to the eyes. I have a solution that nicely colors the prompt, emphasizing activated virtual environments too. See the end of this blog post for more info. Here is a screenshot of the colorized prompt.

Now, let’s see how to activate and deactivate the virtual environment:

jabba@jabba-uplink:~/webapps/hello_project$ ls -al
total 8
drwxrwxr-x 2 jabba jabba 4096 Aug  4 16:04 .
drwxrwxr-x 7 jabba jabba 4096 Aug  4 16:04 ..
jabba@jabba-uplink:~/webapps/hello_project$ workon hello_project
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ ls -al
total 8
drwxrwxr-x 2 jabba jabba 4096 Aug  4 16:04 .
drwxrwxr-x 7 jabba jabba 4096 Aug  4 16:04 ..
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ deactivate 
jabba@jabba-uplink:~/webapps/hello_project$ ls -al
total 8
drwxrwxr-x 2 jabba jabba 4096 Aug  4 16:04 .
drwxrwxr-x 7 jabba jabba 4096 Aug  4 16:04 ..

That is, “workon PROJECT_NAME” activates the virtual environment whose name is PROJECT_NAME. Since we’ve chosen the same name for the Django project’s main folder (1st tier) and for the virtual environemnt, we don’t need to think “damn, what was the name of the corresponding virt. env.?”.

Let’s see what Python packages are installed in this virt. env.:

(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ pip freeze --local

Nothing. That’s normal, we have a brand new virt. env. It’s time to create a basic Django application:

(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ pip install django
Downloading/unpacking django
  Using download cache from /home/jabba/.pip/download_cache/
Installing collected packages: django
Successfully installed django
Cleaning up...
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ startproject hello_project
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ pip freeze --local >requirements.txt
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ cat requirements.txt 
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ mv hello_project/ hello
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ ls -al
total 16
drwxrwxr-x 3 jabba jabba 4096 Aug  4 16:32 .
drwxrwxr-x 7 jabba jabba 4096 Aug  4 16:04 ..
drwxrwxr-x 3 jabba jabba 4096 Aug  4 16:31 hello
-rw-rw-r-- 1 jabba jabba   14 Aug  4 16:32 requirements.txt
(hello_project)jabba@jabba-uplink:~/webapps/hello_project$ cd hello
(hello_project)jabba@jabba-uplink:~/webapps/hello_project/hello$ ls -al
total 16
drwxrwxr-x 3 jabba jabba 4096 Aug  4 16:31 .
drwxrwxr-x 3 jabba jabba 4096 Aug  4 16:32 ..
drwxrwxr-x 2 jabba jabba 4096 Aug  4 16:31 hello_project
-rwxrwxr-x 1 jabba jabba  256 Aug  4 16:31

First, we install Django. Whenever you work with a project, the first thing should be to activate the virtual env. The package django was installed inside the virt. env. With the command “pip freeze --local >requirements.txt” we create the “requirements.txt” file. Whenever you install or uninstall a package, don’t forget to update the “requirements.txt” file using the command above. The command “ startproject hello_project” creates a project called “hello_project“. Next to the “requirements.txt” file you will have a “hello_project” subfolder. This folder is the 2nd tier in our three-tiered approach. This is actually the main folder of a Django project and it can be called anything. I prefer to remove the “_project” suffix from it, that’s why I renamed it to “hello“. Inside this folder you will have a file called ““. Next to ““, there is a subfolder also called “hello_project“. This “hello_project” folder is the 3rd tier and it contains –among others– the “” file, which is the main configuration file of a Django project. The current directory structure should look like this:

(hello_project)jabba@jabba-uplink:~/webapps$ tree
├── hello_project                     (1st tier)
│   ├── hello                         (2nd tier)
│   │   ├── hello_project             (3rd tier)
│   │   │   ├──
│   │   │   ├──
│   │   │   ├──
│   │   │   └──
│   │   └──
│   └── requirements.txt

A Django project usually contains several applications. Each application is stored in a folder that is under the 2nd tier, i.e. next to ““. Normally, each application has a “” file too. I like to call these files “local”, meanwhile the files in the 3rd tier are “global”, since they contain the configuration of the whole Django project. This is why I prefer to add the “_project” suffix to the 3rd tier: I can see immediately which folder is the “global” folder with the main settings.

As seen before, I renamed the 2nd tier to “hello“. This way I can see easier where I am, in which folder. If I had three subfolders with the same name, I should always think a bit “Err, where am I now?”.

OK, you have a basic Django project that works but doesn’t do anything interesting. Well, let’s see what it produces:

(hello_project)jabba@jabba-uplink:~/webapps/hello_project/hello$ ./ runserver

By default, it starts the project on port 8000. Open in your browser. You should see a welcome page.

I’ve already written a bit about “requirements.txt” above. To put it in a nutshell: always keep your “requirements.txt” up-to-date so that it contains the Python packages that are installed in the virt. env.

# create, update
pip freeze --local >requirements.txt    # 1st command

# install from
pip install -r requirements.txt         # 2nd command

Example: you create a project on your home machine. You install some packages and using the first command you update the “requirements.txt” file. Then, at your workplace, you want to polish a bit your pet project. Dropbox synchronized the project but the virt. env. is missing. Set up virtualenvwrapper, create a new virt. env. for the project (as explained at the beginning of this post), then use the second command to install the necessary packages. Voila, you are ready to continue developing your project.

virtualenvwrapper #2
I want to show you a trick how to activate/deactivate a virt. env. easily. When I want to work on a project, first I navigate to the project’s folder. Either I go to the 1st tier (in this example, “~/webapps/hello_project“) or the 2nd tier (“~/webapps/hello_project/hello“). Here, with the simple command “on“, I can activate the virt. env. Similarly, with the command “off” it can be deactivated. To have this functionality, add the following lines to your .bashrc file:

    workon $f
    if [ $? -ne 0 ]; then
        a=`dirname $bak`
        workon $f
        cd $bak
alias on=on_function
alias off='deactivate'

The following condition must meet to make it work: the name of the 1st tier’s folder must be the same as the name of the virt. env. (here we called both of them “hello_project“).

Hmm, this post became a bit long. The next ones will be shorter :)