JBoss JMX Console Vulnerability – Standard Security Is Not Enough !

On 20th October 2011 JBoss released a Security Alert, informing about the existence of a worm which makes use of a security loophole in JBoss JMX Console to attack servers out there in the web. According to this notice, users running unsecured JMX consoles were vulnerable to this attack.

I’ve been running several JBoss Application Server instances exposed to the web, but I always ensure that JMX Console and other management features of JBoss are secured before exposing it to the web. I usually use the standard Username/Password login module for authentication for these JBoss services (I know it’s not very secure, but that was sufficient). Initially when I was setting this up, I referred to this article from JBoss : http://community.jboss.org/wiki/SecureTheJmxConsole [Note: Now it is updated to include the additional steps to protect against this threat].

According to the security alert, password protected JMX consoles were safe from this threat. Since it was password protected, I thought I was secure against this threat. I couldn’t have been more wrong, and I had to learn it the hard way.

Few weeks back, one of my public servers started to show a strange behavior. At certain times, I was not able to access the application via web. When I checked the server health, everything looked normal. Nothing special in the log files, and CPU utilization and memory utilization also looked normal. But server was not responding to HTTP requests. When I took a stack dump, I saw that most of my HTTP threads seem to be busy. This particular application was not heavily used (maximum 20 concurrent users, and about 10 processes which keeps polling the server once in every few minutes), and I had no issues with it until this time. When I restart the server, everything goes back to normal, but after some time (few hours, or few days) it halts again.

Few days back I saw that inside the bin directory of JBoss installation, there are some unknown perl scripts. I read through these scripts, and I saw that one of these scripts are targeting the JMX Console of JBoss. Then I realized that I’m looking at the worm mentioned in the JBoss security alert, and that my server has been a victim of this attack.

I was surprised to see this because I was so sure that my JBoss JMX Console was secured. I checked immediately, and yes, the HTTP Basic Authentication password popup came up as expected. Then I thought may be this is a new variation of the attack which uses a method like brute-force attacks to get the password from JBoss (which would also answer the heavy utilization of HTTP threads).

Later on, when I read the perl scripts closely (a file named linda.pl), I saw that the perl script does a HTTP HEAD request to JMX Console’s Deployer operation. This request is packed with the content of a WAR file which is named as ‘iesvc.war’. Once this request is processed, JMX Console is going to create this file and deploy it under the URL /iesvc. Then the script makes calls to this newly deployed WAR file to do the dirty work for it.

Virus Script - linda.pl

Then it hit me that this guy is using HTTP HEAD specifically to make this call. I double checked my JMX Console’s web.xml file, and then I found the loop-hole!

The standard JMX Console security configuration specifically limits the security to HTTP GET and POST methods. So the attacker is capable of making a HTTP HEAD call as above to infiltrate into JBoss, even if it is password protected. When I checked online for this, I saw that JBoss has already discovered this and updated their instructions to securing the JMX console to include more steps to wipe out this HTTP GET / POST method restrictions. The initial security alert on 20th did not mention about this and I don’t remember seeing this in the Securing JMX Console page on that day. May be I missed it back then, but the page has been last updated on 23rd October, three days after security alert release.

So I guess there could be more people out there like me who are falsely believing that they are safe because they have enabled the standard JBoss security for JMX Console. For those guys, what you should be doing is the following (assuming that you have already enabled security for JMX console; if not follow this article from JBoss). I am using JBoss 4.2.3.GA, and the below instructions are based on that. If you are using a different version, it should be similar.

First of all, in %JBOSS_HOME%/server/%YOUR_SERVER_PROFILE%/deploy/jmx-console.war/WEB-INF/web.xml, delete the two lines which specifies the HTTP methods for security. This will ensure that security will be applied to all HTTP methods.

You should look for these two lines and delete those:

<http-method>GET</http-method>
<http-method>POST</http-method>

Once that is done, your JMX-Console is secure. But I would advise you to take another measure to protect yourself against any future variations of this attack, including brute-force attacks to guess your passwords. That measure is to rename your jmx-console.war file to something else.

The script has the following lines:

$zecmd = "HEAD /jmx-console/HtmlAdaptor?action=invokeOpByName&name=jboss.admin%3Aserv..."

So they are targeting the attack specifically on the /jmx-console/ URL. If you rename this to something else, the attack will not work. Also, guessing which URL JMX console runs is not that easy once you change it. So I would strongly recommend doing that as well.

That’s it for securing JMX console against this attack, but I would also like to mention some interesting things that I discovered in the scripts. I am not a perl developer, and below is my interpretation of the execution of the application based on what I can understand.

First of all, it creates following files in the /bin folder of JBoss.

• bm.c – Boyer-Moore string search algorithm implementation (http://en.wikipedia.org/wiki/Boyer–Moore_string_search_algorithm)
• bm.h – Header file for Boyer-Moore C program
• flu.pl – Core of the Worm. This script obtains information about the victim, and passes it to an IRC channel.
• install-sh – This is a installation script by MIT (1991)
• ipsort – Sorts based on IPv4 addresses
• linda.pl – This is the main execution logic of the worm. Also does the spreading of the worm through network
• Makefile – Script for make command to build the necessary components
• pnscan.c – This is a freeware Parallel Network Scanner by Peter Eriksson (2002).
• version.c – Doesn’t seem to do anything much. It just delcares a character variable with the value 1.11

Also, it creates a set of tar ball files named as kisses.tar.gz, which contains these files.

linda.pl seems to be the main script which does the job. Once it starts execution, it executes ‘make’ command to build the C programs. Then it executes the flu.pl script, and then it executes the pnscan program (created using pnscan.c file) to scan for any more IP addresses in the network. Once it discovers any more JBoss servers running, it attempts to invoke the JMX-Console to infect that server as well.

The flu.pl file refers to localhost, irc.purchaseservice.com and bneth4x.no-ip.org addresses. It executes uname -a and whoami to get information about the victim machine. Then it passes this information to an index.php file running on the above servers. Also from what I can understand, it also passes this information as a private message to a channel in IRC named #jb.

Apart from the execution of the application, I also did some observations about the author of the script. The author of the program seems to be from a non-english speaking region, unless he / she deliberately attempts to make it look like that. The reason for saying that is because in linda.pl, the author has made a comment saying “# Short and unefficiant poc…” instead of “inefficient”. Also, the author refers to the process using a variable named “processo”.

So in conclusion, if you just enable the JMX Console security configuration shipped with JBoss and believe your are safe, you are not. You need to ensure that the security is applied to all HTTP methods. This particular worm makes use of that loophole, and it spreads rapidly across the Internet because it is capable of making use of the victim to infect other machines, which is known as “epidemic algorithms”, because it spreads like a virus.

If you are a JBoss user, make sure that you are protected against this type of attacks, because it could bring down all of your servers with in a matter of seconds. Luckily in my case, each of my servers were isolated via VLANs, and only one machine was infected.

From http://blog.yohanliyanage.com/2011/11/jboss-jmx-console-vulnerability-standard-security-not-enough/