Hosting a Python Django project using uWSGI and nginx

How to setup a webserver to host a Django project? Let’s go through the details of Python hosting process – using uWSGI to host a website based on Django framework.

We will use nginx and uWSGI. Also you’ll have to install PIP and Virtualenv-wrapper, as well as Python dev packages.

I’ll assume that you use Ubuntu Linux, but with small changes you could use this guide on any Linux system.

How uWSGI works

A web server faces the outside world. It can serve files (HTML, images, CSS, etc) directly from the file system. However, it can’t talk directly to Django applications; it needs something that will run the application, feed it requests from web clients (such as browsers) and return responses.

A Web Server Gateway Interface – WSGI – does this job. WSGI is a Python standard.

uWSGI is a WSGI implementation. In this tutorial we will set up uWSGI so that it creates a Unix socket, and serves responses to the web server via the WSGI protocol. At the end, our complete stack of components will look like this:

the web client <-> the web server <-> the socket <-> uwsgi <-> Django

Preparing to install uWSGI

Installing Python packages

You need Python dev packages. To get it on Ubuntu server, run

apt-get install python-dev

Also install PIP and virtualenv-wrapper.

Create a virtual environment for Python project

Make sure you are in a virtualenv for the software we need to install (we will describe how to install a system-wide uwsgi later):

virtualenv uwsgi-tutorial
cd uwsgi-tutorial
source bin/activate

or using virtualenv-wrapper:

mkproject uwsgi-tutorial

Install Django

Install Django into your virtualenv, create a new project, and cd into the project:

pip install Django
django-admin.py startproject mysite
cd mysite

Domain and port

In this tutorial we will call your domain example.com. Substitute your own FQDN or IP address.

Throughout, we’ll using port 8000 for the web server to publish on, just like the Django runserver does by default. You can use whatever port you want of course, but I have chosen this one so it doesn’t conflict with anything a web server might be doing already.

Install and setup uWSGI

Install uWSGI in virtual environment

pip install uwsgi

Of course there are other ways to install uWSGI, but this one is as good as any. Remember that you will need to have Python development packages installed. In the case of Debian, or Debian-derived systems such as Ubuntu, what you need to have installed is pythonX.Y-dev, where X.Y is your version of Python.

Check that uWSGI works

Create a file called test.py:

# test.py
def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    return "Hello World"

Run uWSGI:

uwsgi --http :8000 --wsgi-file test.py

The options mean:

• http :8000: use protocol http, port 8000
• wsgi-file test.py: load the specified file, test.py

This should serve a ‘hello world’ message directly to the browser on port 8000. Visit:

http://example.com:8000

to check. If so, it means the following stack of components works:

the web client <-> uWSGI <-> Python

Check that Django works

Now we want uWSGI to do the same thing, but to run a Django site instead of the test.py module.

If you haven’t already done so, make sure that your mysite project actually works:

python manage.py runserver 0.0.0.0:8000

And if it that works, run it using uWSGI:

uwsgi --http :8000 --wsgi-file barons/wsgi.py

• module mysite.wsgi: load the specified wsgi module

Point your browser at the server; if the site appears, it means uWSGI is able serve your Django application from your virtualenv, and this stack operates correctly:

the web client <-> uWSGI <-> Django

Now normally we won’t have the browser speaking directly to uWSGI. That’s a job for the webserver, which will act as a go-between.

Install nginx

nginx installation on Ubuntu

sudo apt-get install nginx
sudo /etc/init.d/nginx start    # start nginx

And now check that the nginx is serving by visiting it in a web browser on port 80 – you should get a message from nginx: “Welcome to nginx!”. That means these components of the full stack are working together:

the web client <-> the web server

If something else is already serving on port 80 and you want to use nginx there, you’ll have to reconfigure nginx to serve on a different port. For this tutorial though, we’re going to be using port 8000.

Setup a website in nginx

Download uwsgi_params from https://github.com/nginx/nginx/blob/master/conf/uwsgi_params, or just create it and copy and paste following text :

uwsgi_param QUERY_STRING $query_string;
uwsgi_param REQUEST_METHOD $request_method;
uwsgi_param CONTENT_TYPE $content_type;
uwsgi_param CONTENT_LENGTH $content_length;
uwsgi_param REQUEST_URI $request_uri;
uwsgi_param PATH_INFO $document_uri;
uwsgi_param DOCUMENT_ROOT $document_root;
uwsgi_param SERVER_PROTOCOL $server_protocol;
uwsgi_param HTTPS $https if_not_empty;
uwsgi_param REMOTE_ADDR $remote_addr;
uwsgi_param REMOTE_PORT $remote_port;
uwsgi_param SERVER_PORT $server_port;
uwsgi_param SERVER_NAME $server_name;

Copy it into your project directory. In a moment we will tell nginx to refer to it.

Now create a file called mysite_nginx.conf, and put this in it:

# mysite_nginx.conf

# the upstream component nginx needs to connect to
upstream django {
    # server unix:///path/to/your/mysite/mysite.sock; # for a file socket
    server 127.0.0.1:8001; # for a web port socket (we'll use this first)
    }

# configuration of the server
server {
    # the port your site will be served on
    listen      8000;
    # the domain name it will serve for
    server_name .example.com; # substitute your machine's IP address or FQDN
    charset     utf-8;

    # max upload size
    client_max_body_size 75M;   # adjust to taste

    # Django media
    location /media  {
        alias /path/to/your/mysite/mysite/media;  # your Django project's media files - amend as required
    }

    location /static {
        alias /path/to/your/mysite/mysite/static; # your Django project's static files - amend as required
    }

    # Finally, send all non-media requests to the Django server.
    location / {
        uwsgi_pass  django;
        include     /path/to/your/mysite/uwsgi_params; # the uwsgi_params file you installed
        }
    }

This conf file tells nginx to serve up media and static files from the filesystem, as well as handle requests that require Django’s intervention. For a large deployment it is considered good practice to let one server handle static/media files, and another handle Django applications, but for now, this will do just fine.

Symlink to this file from /etc/nginx/sites-enabled so nginx can see it:

sudo ln -s ~/path/to/your/mysite/mysite_nginx.conf /etc/nginx/sites-enabled/

Check that nginx works

Restart nginx:

sudo /etc/init.d/nginx restart

To check that media files are being served correctly, add an image called media.png to the/path/to/your/project/project/media directory, then visithttp://example.com:8000/media/media.png – if this works, you’ll know at least that nginx is serving files correctly.

It is worth not just restarting nginx, but actually stopping and then starting it again, which will inform you if there is a problem, and where it is.

Test both nginx and uWSGI

Let’s get nginx to speak to the “hello world” test.py application.

uwsgi --socket :8001 --wsgi-file test.py

This is nearly the same as before, except this time one of the options is different:

• socket :8001: use protocol uwsgi, port 8001

nginx meanwhile has been configured to communicate with uWSGI on that port, and with the outside world on port 8000. Visit:

http://example.com:8000/

to check. And this is our stack:

the web client <-> the web server <-> the socket <-> uWSGI <-> Python

Meanwhile, you can try to have a look at the uswgi output at http://example.com:8001 – but quite probably, it won’t work because your browser speaks http, not uWSGI, though you should see output from uWSGI in your terminal.

Use Unix socket instead of network port

So far we have used a TCP port socket, because it’s simpler, but in fact it’s better to use Unix sockets than ports – there’s less overhead.

Edit mysite_nginx.conf, changing it to match:

server unix:///path/to/your/mysite/mysite.sock; # for a file socket
# server 127.0.0.1:8001; # for a web port socket (we'll use this first)

and restart nginx.

Runs uWSGI again:

uwsgi --socket mysite.sock --wsgi-file test.py

This time the socket option tells uWSGI which file to use.

Try http://example.com:8000/ in the browser.

If something wrong

Check your nginx error log(/var/log/nginx/error.log). If you see something like:

connect() to unix:///path/to/your/mysite/mysite.sock failed (13: Permission
denied)

then probably you need to manage the permissions on the socket so that nginx is allowed to use it.

Try:

uwsgi --socket mysite.sock --wsgi-file test.py --chmod-socket=666 # (very permissive)

You may also have to add your user to nginx’s group (which is probably www-data), or vice-versa, so that nginx can read and write to your socket properly.

It’s worth keeping the output of the nginx log running in a terminal window so you can easily refer to it while troubleshooting.

Start a Django project using uswgi and nginx

Let’s run our Django application:

uwsgi --socket mysite.sock --module mysite.wsgi --chmod-socket=666

Now uWSGI and nginx should be serving up not just a “Hello World” module, but your Django project.

Create a uWSGI configuration file

We can put the same options that we used with uWSGI into a file, and then ask uWSGI to run with that file. It makes it easier to manage configurations.

Create a file called `mysite_uwsgi.ini`:

# mysite_uwsgi.ini file
[uwsgi]

# Django-related settings
# the base directory (full path)
chdir           = /path/to/your/project
# Django's wsgi file
module          = project.wsgi
# the virtualenv (full path)
home            = /path/to/virtualenv

# process-related settings
# master
master          = true
# maximum number of worker processes
processes       = 10
# the socket (use the full path to be safe
socket          = /path/to/your/project/mysite.sock
# ... with appropriate permissions - may be needed
# chmod-socket    = 664
# clear environment on exit
vacuum          = true

And run uswgi using this file:

uwsgi --ini mysite_uwsgi.ini # the --ini option is used to specify a file

Once again, test that the Django site works as expected.

Start uwsgi as a daemon

To srart uwsgi in daemon mode, create directory /var/log/uwsgi and add in configuration file:

daemonize=/var/log/uwsgi/yourproject.log

After it uwsgi will use this file to log al messages. To see uwsgi messages, run in console:

tail -f /var/log/uwsgi/yourproject.log

Finish webserver setup

Install uWSGI in system

So far, uWSGI is only installed in our virtualenv; we’ll need it installed system-wide for deployment purposes.

Deactivate your virtualenv:

deactivate

and install uWSGI system-wide:

sudo pip install uwsgi

# Or install LTS (long term support).
pip install http://projects.unbit.it/downloads/uwsgi-lts.tar.gz

The uWSGI wiki describes several installation procedures. Before installing uWSGI system-wide, it’s worth considering which version to choose and the most apppropriate way of installing it.

Check again that you can still run uWSGI just like you did before:

uwsgi --ini mysite_uwsgi.ini # the --ini option is used to specify a file

Emperor mode

uWSGI can run in ‘emperor’ mode. In this mode it keeps an eye on a directory of uWSGI config files, and will spawn instances (‘vassals’) for each one it finds.

Whenever a config file is amended, the emperor will automatically restart the vassal.

# create a directory for the vassals
sudo mkdir /etc/uwsgi
sudo mkdir /etc/uwsgi/vassals
# symlink from the default config directory to your config file
sudo ln -s /path/to/your/mysite/mysite_uwsgi.ini /etc/uwsgi/vassals/
# run the emperor
uwsgi --emperor /etc/uwsgi/vassals --uid www-data --gid www-data

You may need to run uWSGI with sudo:

sudo uwsgi --emperor /etc/uwsgi/vassals --uid www-data --gid www-data

The options mean:

• emperor: where to look for vassals (config files)
• uid: the user id of the process once it’s started
• gid: the group id of the process once it’s started

Check the site; it should be running.

Autostart uWSGI with system start

The last step is to make it all happen automatically at system startup time.

Edit /etc/rc.local and add:

/usr/local/bin/uwsgi --emperor /etc/uwsgi/vassals --uid www-data --gid www-data

before the line “exit 0”.

And that should be it!

What to do next?

It is important to understand that this has been a tutorial, to get you started. You do need to read the nginx and uWSGI documentation, and study the options available before deployment in a production environment.

Both nginx and uWSGI benefit from friendly communities, who are able to offer invaluable advice about configuration and usage.

nginx

General configuration of nginx is not within the scope of this tutorial though you’ll probably want to it to listen on port 80, not 8000, for a production website.

You also ought to consider at having a separate server for non-Django serving, of static files for example.

uWSGI

uWSGI supports multiple ways to configure it. See uWSGI’s documentation and examples.

Some uWSGI options have been mentioned in this tutorial; others you ought to look at for a deployment in production include (listed here with example settings):

env = DJANGO_SETTINGS_MODULE=mysite.settings # set an environment variable
pidfile = /tmp/project-master.pid # create a pidfile
harakiri = 20 # respawn processes taking more than 20 seconds
limit-as = 128 # limit the project to 128 MB
max-requests = 5000 # respawn processes after serving 5000 requests
daemonize = /var/log/uwsgi/yourproject.log # background the process & log

Running uWSGI via Upstart

Upstart is the init system of Ubuntu-like distributions.

It is based on declarative configuration files – not shell scripts of yore – that are put in the /etc/init directory.

# simple uWSGI script

description "uwsgi tiny instance"
start on runlevel [2345]
stop on runlevel [06]

exec uwsgi --master --processes 4 --die-on-term --socket :3031 --wsgi-file /var/www/myapp.wsgi

# Emperor uWSGI script

description "uWSGI Emperor"
start on runlevel [2345]
stop on runlevel [06]

exec uwsgi --master --die-on-term --emperor /etc/uwsgi