Python-Flask-uWSGI-Setuptools: How To Deploy Simple WSGI Applications

05/23/2014

Python-logo-master-v3-TM


Python is a great language for quickly writing applications. The solid standard library coupled with a great community results in a development environment that is fast and iterative. However, I've noticed many stumble after the application is written and ready to be deployed and consumed by users. The documentation and best practices can feel scattered and that is because there are many ways to deploy a Python application and it can get confusing.

With that in mind, I thought it would be helpful to describe a single, repeatable, and painless way to get from a Python application to deployment.

In order to follow along with the example, you may need to farmiliaize yourself with git in order to fetch the example code. If you already have git installed and configured, then continue onward.

The Bits and Pieces

Here's a quick run through of the pieces we will work with

WSGI

First, a bit of acronym trivia. WSGI stands for Web Server Gateway Interface and was the Python community's attempt at standardizing the API for applications and web servers to talk to each other. Before the WSGI spec, the way that Python applications and servers spoke could differ in each implementation.  Applications often needed to be designed for a specific API and portablity was harder to achieve.  Want to move over to use FastCGI or mod_python?  Have fun re-configuring your application.  Now, most modern Python web frameworks support an export of the application in a way that is consumable by a WSGI server.

Standards. Good. Moving on.

Flask

Flask is a microframework that supports the WSGI standard.  The project's philosophy is focused on extensibility and modularity.  It comes out-of-the-box with less features than a more monolithic framework, but instead defines a solid extension API.  This is what we use to write the application and abstracts away some of the repetitive coding tasks like routing URLs to functional bits of code.

uWSGI

We have our application code and now we need a way to serve it to users.  uWSGI is an application server that supports the WSGI specification.  It accepts a defined configuration and a packaged application that also supports WSGI.  It can be run as a standalone server or behind a web server like Apache or nginx.

setuptools

A third-party library used for zipping up Python packages and modules into a single distributable. This can be a library, tool, or in our case, an application. Python's setuptools has a reputation for being tough on beginners. The options are many and the behavior can sometimes seem erratic to those that are new to Python packaging and distribution. We are going to use a very basic setuptools configuration. Setuptools configuration is commonly placed into a project's setup.py file.

The App

Before writing this post, I've set up a small flask application that I will be using as an example.  You can get the code at https://github.com/chroto/flaskplate.  In order to run, setup, and install properly, you will need the following dependencies:

If you are not farmiliar with virtualenv or Fabric, the quick explanation is that they will handle the orchestration of installing and running the app with the proper environment.

Setup and Install

Start by cloning the application code and running fabric:

[me@localhost] $ git clone https://github.com/chroto/flaskplate.git
[me@localhost] $ cd flaskplate
[me@localhost] $ fab setup 

Running fab setup will create a virtualenv to install the app and its dependencies in the env directory. It will also create a distributable Python .egg based on the configuration in the setup.py file.

[me@localhost] $ ls dist 
flaskplate-0.1-py2.7.egg 

Python "eggs" are simply ZIP archives that have Python-related metadata. You could actually unzip that flaskplate-0.1-py2.7.egg and you would simply get a directory very similar to the one you are currently sitting inside.

After this egg is created in the local dist directory, it is copied into the activated Python environment's library and is now available using the import function from any Python script.


    from setuptools import setup

    project = "flaskplate"

    setup(
        name=project,
        version='0.1',
        description='Example Flask application',
        author='Chris Proto',
        author_email='me at mydomain.com',
        packages=[
            'flaskplate',
        ],
        include_package_data=True,
        zip_safe=False,
        install_requires=[
            'Flask',
            'Flask-Script'
        ]
    )

This simple setup.py file defines the package flaskplate with some metadata like author, email, description. It also includes parameters for include_package_data which tells setuptools to include non-python files ( so we can package up our HTML and CSS ).

There are optimizations in Python eggs that allows it to read .py files even from a compressed format (think of Java's JAR format), but since we have data files ( HTML and CSS again ) we need to tell setuptools to turn off this behavior with zip_safe=False

Running the application

[me@localhost] $ fab run 
[localhost] local: python manage.py run
 * Running on http://127.0.0.1:5000/
 * Restarting with reloader  

Navigate to http://127.0.1:5000/ and in your browser of choice and you should see the following:

flaskplate

Configuration and Deployment

For this task, we will install uWSGI in the same virtualenv environment as our application. First, we activate the virtualenv:

[me@localhost] $ source ./env/bin/activate 
(env) [me@localhost] $ 

If we have activated the environment, the prompt should have changed to start with (env). Next, we install uWSGI:

(env) [me@localhost] $ pip install uwsgi

We now have the application server installed and ready to be configured.

For this step, you will need to ensure that you have a C compiler installed. On Linux or MacOSX, you can determine if this is the case by checking the CC environment variable.

Configuring uWSGI

Included in the fabfile is a command to set up a contained deployment environment.

(env) [me@localhost] $ fab setup_deploy 

Most of what is built out are directories that uWSGI will drop things like process ID files (PIDs), logs, and socket files. However, there is also a basic configuration file that is created at env/etc/uwsgi/flaskplate.ini. This config tells uWSGI what application to run and how.

flaskplate.ini

[uwsgi] 
# Tell uWSGI to run from a certain directory.
chdir=/Users/chris.proto/apps/example_apps/flaskplate 
# Listen on port 9000 
http-socket=:9000 
# The Python module that contains an application object to deploy
module=flaskplate.wsgi 
# The WSGI supported application object 
callable=application 
# How many uWSGI processes to spawn 
processes=1 
# How many threads to spawn per process 
threads=2 
# Python home directory. It is the virtual environment we have created. 
home=/Users/chris.proto/apps/example_apps/flaskplate/env 

Once uWSGI is configured and set up, we can start up the uWSGI master and deploy the application.

(env) [me@localhost] $ fab deploy

This command will run uWSGI as a daemon and log events into the log directory that fab setup_deploy had created. The default setup will tell uWSGI to listen on port 9000. To confirm the that it is indeed running navigate to http://127.0.0.1:9000/

Screen Shot 2014-05-22 at 3.24.51 PM

What's Next?

Hopefully, this can be used as a good starting point for formulating a sane deployment strategy for your own services and applications. I recommend the next step is to read the uWSGI configuration documentation. There you'll find many options for configuring uWSGI in any number of ways. For example, uWSGI can be run in a cluster or behind a webserver like nginx or Apache for added robustness.

Whatever your needs may be, the uWSGI-Flask stack may be a good solution to power your next application or service.

At Craftsy, we often need to spin up internal applications and services  to support dynamic business and engineering concerns.   These applications must focus on ease of deployment and quick, iterative development. For these reasons, a method similar to the one described in this post has been used in our infrastructure with the only major difference being most of this process has been automated away using management tools like Puppet

Comments (0)

The comments to this entry are closed.