First Level of Support With Monit

Monit is a brilliant bit of software that monitors if your services are still up, and it can reload them if they die. This definitely helps with the first level of support when your servers get a little bit overworked. It can also check that the filesize, checksum, and the permissions of the program remained the same.

Installation

The easiest way to install Monit is using apt-get:

sudo apt-get install monit

Configuration

If you want to go with the default configuration, it will run every two minutes, and you can view the logs of what was reloaded and when in /var/log/monit.log.

To edit those settings, please view the following configuration file:

sudo nano /etc/monit/monitrc

It runs as a service and can be started using the following command:

sudo service monit start

Monitoring Programs

Below, you can find examples of the configuration for nginx and php that will live in /etc/monit/conf.d/

nginx

check process nginx with pidfile /var/run/nginx.pid
 group webserver
 start program = "/etc/init.d/nginx start"
 stop program = "/etc/init.d/nginx stop"
 if failed host 127.0.0.1 port 80 protocol http
 and request "/robots.txt"
 then restart
 if 3 restarts within 5 cycles then alert

php

check process php5-fpm with pidfile /var/run/php5-fpm.pid
 group webserver
 start program = "/etc/init.d/php5-fpm start"
 stop program = "/etc/init.d/php5-fpm stop"
 if failed
 port 80
 protocol http
 request '/index.php'
 then restart
 if 3 restarts within 5 cycles then alert
 depends on nginx

Finish Up

You can add more configuration and you can even add alerting via email or pager duty to let you know when a service fails too many times.

I have used this successfully for a while now and it saved me a lot of time, as well as helping to fix hard to debug problems with applications and server behavior.