Deployment

Deployment

Deployment covers the specific action of setting a site up that uses our Django configuration. This example is done with the respect site but should apply with appropriate naming to every installation.

This assumes that the server is installed. There is some overlap between the installation instructions which include details for a local installation and these deployment notes.

note: In order to keep everything tidy in the repository, there is a top level folder called deploy which contains the specific deployment files for a given deployment. These are the nginx configuration file, the upstart job file, a (shared) gunicorn configuration file and the demon script used to launch gunicorn from the upstart job.

One day I'll probably combine the demon script with the upstart job but upstart jobs are notoriously difficult to debug so I've separated them for the time being to keep the upstart job as simple as possible.

On another day I hope to create a fabric script so that there can be a single set of 'template' deploy files that are populated using django settings

Outline

We create a user that can pull the code from the repository and that will have the necessary access rights to run the service (but no more).

We create the required database user and database.

We create one or more working copies of the project in the user's home directory and link up the necessary nginx and upstart jobs to ensure that the site is served and the required processes restart when the server is restarted.

Procedure

Create User

The username is chosen to reflect the primary name of the service. This will often be a simple abbreviation of the domain name.

Add the user using adduser <username>. Default everything except the password.

Create the user keys

The user requires an identity to use for the deployment key used to obtain the working copies from git.

respect:~ $ mkdir .ssh
respect:~ $ cd .ssh
respect:~/.ssh $ ssh-keygen -t rsa -C "paul.whipp@gmail.com"
<default everything. Do not use a passphrase>
respect:~/.ssh $ cat id_rsa.pub
ssh-rsa AAAA...5Tt15 paul.whipp@gmail.com

Copy the displayed key into github as a deploy key. This grants the user read access so that it can pull from git.

Create the working copy

Each site needs a working copy of the source. We use a working copy rather than a simple copy of the source so that updates can take full advantage of git's compression and patching.

The working copy is cloned into a working copy whose root folder name reflects the use to which the working copy will be put. Thus for the live, production site we use the name 'production', for a staging site we use 'staging', for a continuous integration site we use 'ci'.

respect:~/.ssh $ git clone git@github.com:cccs-web/respect.git production
...

Create the virtualenv

The virtual environment contains the python modules used by the site and is used to conveniently specify the default settings that are used.

By convention we name the virtualenv with the same name used for its associated working copy. In some cases (such as using alternative python file versions or settings files but the same working copy), suffixes may be added to clarify the name.

note: The staging site uses its own database (hence has its own settings) and its own branch ('staging'). Thus we create a separate working copy and virtual env for it.

respect:~ $ mkvirtualenv productionNew python executable in production/bin/python
Installing setuptools, pip...done.
(production)respect:~/production $ setvirtualenvproject
Setting project for production to /home/respect/production
(production)respect:~/production $ add2virtualenv .
Warning: Converting "." to "/home/respect/production"
(production)respect:~/production $ echo "export DJANGO_SETTINGS_MODULE=respect.settings.production" >> ~/.virtualenvs/production/bin/postactivate 
(production)respect:~/production $ deactivate
respect:~/production $ workon production

setvirtualenvproject is a convenience so that workon switches us to this folder. add2virtualenv puts our modules on the python path. The echo statement gets our settings defaulted so that all the django commands work nicely.

note: I also alias django=django-admin.py in the .bashrc file for convenience.

Now we can populate the virtualenv:

(production)respect:~/production $ pip install -r requirements.txt 
...
Successfully installed Mezzanine ipython psycopg2 gunicorn mezzanine-slides setproctitle bleach requests-oauthlib django tzlocal filebrowser-safe future pillow requests beautifulsoup4 grappelli-safe six html5lib oauthlib pytz
Cleaning up...

Pay attention to the information. If the pip install fails then the server is probably not installed correctly.

Set up the database

The site requires an an appropriate database and database user.

Obtain the secrets.py file

The user credentials are not stored in the repository so if you intend to work with an existing database, you must obtain an appropriate secrets.py file for it and use its contents below. If you are creating a new database, create and edit your secrets.py file (with random stuff of your choosing) thus:

(production)respect:~/production $ cp respect/settings/includes/secrets.py.example respect/settings/includes/secrets.py
(production)respect:~/production $ # Edit the secrets.py to add your values

Bring in any required media

Media files are not stored in the repository because they are potentially added by users on the site. Copy any required existing media files to the media folder. e.g.

(production)respect:~/production $ tar -xzf ../respect.media.tar.gz 

Create the database user

Create the database user specified in the settings. These commands assume that you are a postgres superuser.

~ $ psql -c "CREATE USER username WITH PASSWORD 'password';"
CREATE ROLE

Obtain or create the database

If you are creating a new database, execute the following command as the postgres user (use the settings to establish the correct database name):

~ $ psql -c "CREATE DATABASE database_name WITH OWNER username;"

If you have the database, restore it with the following command as the postgres or an appropriate user:

~ $ pg_restore -d postgres -C respect.pg_dump 

Collect Static Files

Collect the static files.

(production)respect:~ $ django collectstatic
...

Set up upstart

Copy the relevant upstart configuration file to /etc/init. This must be done as root. The name of the job uses username_siterole:

~ # cp /home/respect/production/deploy/production/upstart.conf /etc/init/respect_production.conf
~ # service respect_production start
respect_production start/running, process 22882

Set up nginx

Link the relevant nginx configuration file up appropriately

~ # ln -s /home/respect/production/deploy/production/nginx /etc/nginx/sites-available/respect_production
~ # ln -s /etc/nginx/sites-available/respect_production /etc/nginx/sites-enabled/
~ # service nginx restart
 * Restarting nginx nginx                                                                     [ OK ] 

All done!