Over a million developers have joined DZone.

Detect Suspicious Linux Processes

Ever bothered by suspicious processes running on your servers? No doubt how dangerous they might be: valuable data leaked, CPU/memory wasted, or DDoS attack other victims, etc. Read this article to learn how to easily capture those annoying troublemakers.

· Performance Zone

Discover 50 of the latest mobile performance statistics with the Ultimate Guide to Digital Experience Monitoring, brought to you in partnership with Catchpoint.

Ever bothered by suspicious processes running on your servers? No doubt how dangerous they might be: valuable data leaked, CPU/memory wasted, or DDoS attack other victims, etc. Want to learn how to easily capture those annoying troublemakers? Even better, learn how to get alerted without additional human effort.

List all non-kernel processes. Usually, kernel processes are safe and clean. For kernel processes, either PID(process id) is 2 or PPID(parent process id) is 2. Here is how to get all non-kernel processes.

# rss(resident set size): real RAM usage
# -deselect: rule out matched processes
root@denny:~# ps --ppid 2 -p 2 -p 1 \
   --deselect -o uid,pid,rss,%cpu,command
UID   PID   RSS %CPU COMMAND
   0   411  1848  0.0 /lib/systemd/systemd-
   0   572  2904  0.0 dhclient -1 -v -pf /r
 102   902  1244  0.0 dbus-daemon --system
   0   912  1948  0.0 /lib/systemd/systemd-
   0  5869   388  0.0 upstart-socket-bridge
 200  1953   904  0.0 /usr/sbin/apache2 -k
 200  3463  3700  0.0 /usr/sbin/apache2 -k
  ...  ...
  ...  ...
   0  5098  4224  0.0 sshd: ubuntu [priv]
   0  5139  1748  0.0 /usr/bin/python /usr/
 200  5140  3484  0.0 /usr/bin/python /usr/
 200  5176  1904  0.0 sshd: ubuntu@pts/3
 200  5177  3860  0.0 -bash
 200  5193  1200  0.0 tmux attach -t denny
   0  5297  4224  0.0 sshd: ubuntu [priv]
  ...  ...
  ...  ...

Rule out trusted processes. We may have many processes running, which are expected and trusted. e.g apache2, tomcat7, mysqld, etc. To avoid distraction, build a white list especially for your project.

Sort processes by memory and CPU. We're more concerned about suspicious processes using noticeable resources.

# Sort by memory first, then cpu
ps --ppid 2 -p 2 -p 1 --deselect \
  -o uid,pid,rss,%cpu,command, \
  --sort -rss,-cpu

Automate Detection Process and Get Alerts. We hide all the complexities and whitelist configuration in a python script (detect_suspicious_process.py). If you issue the python command, you may see output like "Identified processes count: XXX." Define a scheduled task to run periodical checks and confirm the number.

If the number is not 0 or it changes, send alerts. It might take a while to build a suitable white list. Once it's done, your servers are always more secured and managed!

wget -O /tmp/detect_suspicious_process.py \
https://raw.githubusercontent.com/\
DennyZhang/devops_public/tag_v2/python/\
detect_suspicious_process/\
detect_suspicious_process.py

# Detect suspicious process
python /tmp/detect_suspicious_process.py

# Detect by customized whitelist
python /tmp/detect_suspicious_process.py \
   --whitelist_file /tmp/whitelist.txt

Here is some more reading that you can do: Automate Insecure Ports Check By Nmap.

Is your APM strategy broken? This ebook explores the latest in Gartner research to help you learn how to close the end-user experience gap in APM, brought to you in partnership with Catchpoint.

Topics:
linux ,devops ,security

Published at DZone with permission of Denny Zhang, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}