Pgrep and Pkill: Linux Scripting and Process Management Friends.

Often when inspecting a UNIX server as a sysadmin you need to find and kill a (hung) process running with certain command line arguments.

The traditional solution is to search through ps process listing outputs using the grep command:

ps -Af|grep -i "xcmd"
... list of all processes running on the server matching xcmd here ...

Even though chaining commands is something UNIX philosophy encourages, there exist specific commands for finding processes and killing them. pgrep searches from processes and pkill kills them. These sister commands use the same syntax.

Here is an example. We have a cron script which will open a SSH tunnel for the background. Then this tunnel is used to communicate with a remote server by other processes in the same script. Sometimes, however, SSH tunnel can hang due to network reasons. The tunnel is started with command and this is used in one of /etc/cron.d scripts:

# daemon mode + do not open remote shell
ssh -fN example.com -L 2244:internal.example.com:22

(If you are planning to create more robust solution it is suggested to use lock files instead where the process id is written each time the background process is started. However, here we discuss about the solution which is quick to implement and easy to understand.)

First we note this problem by inspecting the process list by the traditional means:

ps -Af|grep -i ssh

And in the output we see the bad process – we verify it is not working as the tunneled port indeed does not communicate with the remote server:

root      8292     1  0 09:37 ?        00:00:00 ssh example.com -L 2244:internal.example.com:22 -fN

Now we can see if pgrep also can find this process by matching full command line (-f switch):

pgrep -f 'ssh example.com'
8292

And now, we simply put a command at the beginning of our cron script to kill the previous tunnel, regardless if it is hung or not, when running the script. Please note that the side effect is that this will kill all other SSH commands with the same command line on the server: in our case it doesn't matter, as the tunnel should be only started and maintained by this one cron script.

#!/bin/sh

# Kill existing pipes if they are hung
pkill -f 'ssh example.com'

ssh example.com -L 2244:internal.example.com:22 -fN

# script goes here...