Deploying Graphite on Mongrel2

Spike Morelli

May. 19, 12 · Interview

Like (0)

Save

6.79K Views

Introduction

I wanted to try out a new webserver and chose mongrel2. To get hyped about mongrel2, read about its features and then try the quick start guide.

I was also setting up graphite on the same server and previously used apache for that. So I could just proxy all graphite requests from mongrel2 to apache, or I could do try to get mongrel2 to talk wsgi to graphite directly. I chose latter and here’s how I did it. Bear in mind that there are other options and if anyone has any suggestion for improvements feel free to share them in the comments!

About the mongrel2 manual

I’ll be linking to bits and pieces of the mongrel2 manual, but note that it’s really a much better read if you take a look at it from the beginning. It is long, but it’s very entertaining and easy to follow. If you haven’t fiddled with mongrel2 before, feel free to deviate from this guide to follow the mongrel2 manual at any point, and getting back later.

With that said, let’s get started!

Installing mongrel2

Follow the instructions in the manual to install mongrel2 and zeromq. You just need to go over sections 2.1, 2.2 and 2.3.

What you’ll end up with after the final sudo make install is the m2sh and mongrel2 binaries in your PATH.

Installing a mongrel2 WSGI handler

Mongrel2 doesn’t talk wsgi, fastcgi or anything else language specific. What it does talk is a simple backend protocol which any language can easily support. I really recommend to at least read the introduction before continuing.

So to get mongrel2 to talk wsgi, you just need a handler which acts as a gateway between that zeromq protocol/format and wsgi.

There are already several such handlers written, here are some:

I chose m2wsgi mostly because someone in #mongrel2 on irc.freenode.org recommended it. Feel free to try whatever matches your requirements.

I installed libraries and handler with pip.

    pip search mongrel2
    sudo pip install m2wsgi

Setting up mongrel2

Let’s set up mongrel2 in /var/www/mongrel2/. Feel free to change that path if you don’t like it. Note that the folder and all its content should be owned by your user, not root.

mkdir /var/www/mongrel2
cd /var/www/mongrel2
mkdir run logs tmp

Mongrel2 uses sqlite3 for configuration. Let’s use the m2sh tool to generate it from a text file. First create and open /var/www/mongrel2/mongrel2.conf with your favorite editor, and put this into it:

content = Dir(base=’/content/’,
index_file=”,
default_ctype=’text/plain’)
wsgi_handler = Handler(send_spec=’tcp://127.0.0.1:9999′,
send_ident=’ebe4ee7d-6a47-42dd-9acd-1707add81835′,
recv_spec=’tcp://127.0.0.1:9998′, recv_ident=”)
routes={
‘/’: wsgi_handler,
‘/content’: content
}
localhost = Host(name=”localhost”, routes=routes)
localip = Host(name=”127.0.0.1″, routes=routes)
main = Server(
uuid=”31bf6b07-a147-466c-87b5-961481b99201″,
access_log=”/logs/access.log”,
error_log=”/logs/error.log”,
chroot=”/var/www/mongrel2/”,
pid_file=”/run/mongrel2.pid”,
default_host=”localhost”,
name=”main”,
port=6767,
hosts=[localhost, localip]
)
settings = {“zeromq.threads”: 1}
servers = [main]

If something raises questions with you when looking over that, read about it in the manual. For now, note that:

The static content/ directory refers to /opt/graphite/webapp/content/. I’ll come back to that in a minute.
The wsgi_handler entry will talk with m2wsgi over zeromq. We’ve specified the ports 9999 and 9998 but you can set these to anything you want.
The server will only respond to requests with http Host header localhost and 127.0.0.1 for now. Add more hosts to the hosts array in Server if you want to be able to refer to the server with something other than localhost when you’re testing it.

Now create the sqlite3 database, and verify its existence:

    cd /var/www/mongrel2
    m2sh load -config mongrel2.conf -db config.sqlite
    ls -l config.sqlite
    -rw-r–r– 1 murray murray 45K Apr 14 17:54 config.sqlite

Looks good!

Start the graphite WSGI app

Assuming you’ve installed graphite in /opt/graphite/, you need to edit graphite.wsgi.

    cd /opt/graphite/conf/
    cp graphite.wsgi{.example,}

Edit graphite.wsgi with your favourite editor, so that it looks something like this:

    import os, sys
    sys.path.append(‘/opt/graphite/webapp’)
    os.environ['DJANGO_SETTINGS_MODULE'] = ‘graphite.settings’
    import django.core.handlers.wsgi
    application = django.core.handlers.wsgi.WSGIHandler()
    from m2wsgi.io.standard import WSGIHandler, Connection
    conn = Connection(send_sock=”tcp://127.0.0.1:9999″,
    recv_sock=”tcp://127.0.0.1:9998″)
    handler = WSGIHandler(application, conn)
    handler.serve()

Nothing is changed from the example except for adding the last 5 lines.

The static content folder

The folder /opt/graphite/webapp/content/ contains static web content for the webapp. I was a little bit unsure how to set this up since the mongrel2 process chroots to its directory, and I chose not to clutter the /opt/graphite/webapp directory by running mongrel2 in it.

I ended up copying /opt/graphite/webapp/content/ to /var/www/mongrel2/content/ and setting up a static route for it in mongrel2.conf.

I don’t know if graphite does something with the content directory other than to refer to it in the HTML. Just in case, i created a symlink to it:

    cp -a /opt/graphite/webapp/content/ /var/www/mongrel2/
    cd /opt/graphite/webapp
    ln -s /var/www/mongrel2/content/

(Remember that it wouldn’t make sense to symlink the other way since the mongrel2 process is chrooted!)

Running

So let’s finally see if we can get this thing running.

Start the python wsgi application. It will just run without outputting anything for now (graphite might output some warnings if you haven’t set up all its config files). Note: my python 2 binary is called python2, you might have to use that instead, depending on your distro and python setup.

    cd /opt/graphite/conf
    python graphite.wsgi

Start mongrel2. It should output several lines about what it’s doing, with the last message being “All loaded up, time to turn into a server”, before daemonizing.

    cd /var/www/mongrel2
    m2sh start -every -sudo

Testing

Let’s verify that it’s running as it should.

    m2sh running -db config.sqlite -name main
    mongrel2 at PID 26089 running.
    ps ax | grep mongrel2
    26089 ?        Ssl    0:00 mongrel2 config.sqlite 31bf6b07-a147-466c-87b5-961481b99201
    sudo netstat -lnp
    # I removed irrelevant lines here…
    tcp  0  0  127.0.0.1:9998  0.0.0.0:*  LISTEN  26089/mongrel2
    tcp  0  0  127.0.0.1:9999  0.0.0.0:*  LISTEN  26089/mongrel2
    tcp  0  0  0.0.0.0:6767    0.0.0.0:*  LISTEN  26089/mongrel2

Looks good! Point your browser to http://localhost:6767/ and you should see graphite running!

Final thoughts

You’re gonna want to check out m2wsgi and its different configuration options, and change /opt/graphite/conf/graphite.wsgi to suit your needs.
Starting the wsgi application manually is probably not an option. Mongrel2 offers a user space process manager called procer which may be just what you need. Note that the manual also mentions a few alternatives) (scroll down to Note 5), and also offers some deployments tips which might be worth checking out.
In the apache vhost example included with graphite (graphite-web-0.9.8/examples/example-graphite-vhost.conf) there’s a media folder pointed to some media folder in Django and a reference to “The Django admin site media”. I haven’t really used that myself so I don’t know what it is, but to make that work you would probably have to do something similar to what I did with the /opt/graphite/webapp/content/ directory above.

Graphite (software)

Opinions expressed by DZone contributors are their own.

Comments