ClassLoaderLocal: How to Avoid ClassLoader Leaks on Application Redeploy

"OutOfMemoryError: PermGen" is a very common message to see after a few redeploys. The reason why it's so common is that it's amazingly easy to leak a class loader. It's enough to hold a single outside reference to an object instantiated from a class loaded by the said class loader to prevent that class loader from being GC-d.

In this post I'll review how we solved this problem in JavaRebel, and share the solution with you. It's not a magical solution, but it will help alleviate some of the problems introduced in both libraries and applications in Java EE.

The most common way to leak is to register some kind of a callback object and never deregister it. E.g. look at the following code:

Core.addListener(new MyListener());

If Core is a part of the framework/platform/container then it will hold to MyListener long after the application was redeployed and the class loader left hanging.

 

Let's see if we can do anything to solve this. The Core implementation looks something like this:

public class Core {
  List listeners = new ArrayList();

  void addListener(Listener l) {
    listeners.add(l);
  }

  void fireListeners() {
    // Exercise for the reader!
  }
}

The problem is that listeners provides a strong reference to the Listener object. What if we replace it by a weak one?

public class Core {
  List listeners = new ArrayList();

  void addListener(Listener l) {
    listeners.add(new WeakReference(l));
  }

  void fireListeners() {
    // Exercise for the reader!
  }
}

Unfortunately although this does solve the problem of GC-ing the class loader, it doesn't really work. The Listener behind the weak reference will be GC-d at first opportunity and after that it'll no longer receive any callbacks. To illustrate why it's a problem the code above is basically equivalent to throwing the reference away altogether:

public class Core {
  List listeners = new ArrayList();

  void addListener(Listener l) {
    // Listener is ignored and GC-d
  }
}

Replacing weak reference with a soft one doesn't improve the situation, just delays the inevitable a bit further. Both are useful for caches, where objects can be recreated at will, but not in this case where we have an externally created object.

So what do we do? What we'd like to do is have the Listener reference to depend on the class loader somehow. Unfortunately, to the best of my knowledge, there isn't a ready-made solution for that, and there's no way to achieve it with any combinations of weak references without causing problems.

What we'd like to have is an ability to add a strong reference to the class loader: in other words have it carry a custom property:

void addListener(Listener l) {
  ClassLoader cl = l.getClass().getClassLoader();
  List lls = (List) cl.getProperty("CoreListeners");
  if (lls == null) {
    lls = new ArrayList();
    cl.putProperty("CoreListeners", lls);
  }
  lls.add(l);
}

That would work, wouldn't it? Well, not quite. We also need to save a reference to the class loaders, so that we could later go over all of them. Here the WeakHashMap is useful:

Map classLoaders = new WeakHashMap();

void addListener(Listener l) {
   //...
  classLoaders.put(cl, Boolean.TRUE);
}

There's not WeakHashSet in Java, so we're just using a boolean flag as the value.

So this would probably work, but unfortunately class loaders don't have a getProperty()/putProperty() API. However, it turns out that with a bit of a hack we can simulate it, by generating a unique class per class loader to hold the properties for us. Let's see how it's done!

We start with a little boilerplate:

class ClassLoaderLocalMap {
  private static Method defineMethod;
  private static Method findLoadedClass;

  static {
    try {
      defineMethod = ClassLoader.class.getDeclaredMethod(
         "defineClass",
          new Class[] { 
            String.class, 
            byte[].class, 
            int.class,
            int.class });
      defineMethod.setAccessible(true);

      findLoadedClass =
        ClassLoader.class.getDeclaredMethod(
          "findLoadedClass", 
          new Class[] { String.class});
      findLoadedClass.setAccessible(true);   
    }
    catch (NoSuchMethodException e) {
      throw new RuntimeException(e);
    }
  }  
} 

This will give us access to ClassLoader protected methods defineClass() and findLoadedClass() later on. Now let's setup the basic API:

public static void put(
  ClassLoader cl, 
  Object  key, 
  Object value) {
  // Synchronizing over ClassLoader is safest
  synchronized (cl) {
    getLocalMap(cl).put(key, value);
  }
}

public static Object get(
  ClassLoader cl,
  Object key) {
  // Synchronizing over ClassLoader is safest
  synchronized (cl) {
    return getLocalMap(cl).get(key);
  }
}

getLocalMap() method should return a map of entries associated with the class loader. How should that work?

Next we introduce a map from class loaders to unique holder class names. We also introduce a nextHolderName() method that generates unique names:

private static final Map classLoaderToHolderClassName = 
  Collections.synchronizedMap(new WeakHashMap())
private static int counter = 1;

private static synchronized String nextHolderName() {
  return "ClassLoaderLocalMapHolder$$GEN$$" + counter++;
}

Finally we can implement the getLocalMap() method (to save space I removed all exception handling):

private static Map getLocalMap(ClassLoader cl) {
  String holderClassName = 
    (String) classLoaderToHolderClassName.get(cl);
  if (holderClassName == null) {
    holderClassName= nextHolderName();
    classLoaderToHolderClassName.put(
      cl, holderClassName);
  }

  Class holderClass =
    (Class) findLoadedClass.invoke(
      cl,
      new Object[] {propertiesClassName});

  if (holderClass  == null) {
    byte[] classBytes = 
      buildHolderByteCode(holderClassName);

    holderClass = (Class) defineMethod.invoke(cl,
        new Object[] {
          holderClassName, 
          classBytes, 
          new Integer(0),
          new Integer(classBytes.length)});
  }

  return (Map) holderClass
    .getDeclaredField("localMap").get(null);
}

The last method to implement is buildHolderByteCode. It's quite trivial and builds the following class renamed to the unique name:

public class ClassLoaderLocalMapHolder$$GEN$$X {
  public static final Map localMap = new HashMap(); 
}

The code can be derived using ASMifier with just a little customization, you can look it up in the full source code.

Although we could now easily implement the original example it makes sense to do just a little bit extra effort and introduce a ClassLoaderLocal, with behavior similar to the ThreadLocal:

public class ClassLoaderLocal {
  private Object key = new Object();
  
  public Object get(ClassLoader cl) {
    if (!ClassLoaderProperties.containsKey(cl, key))
      return null;
    return ClassLoaderProperties.get(cl, key);
  }
  
  public void set(ClassLoader cl, Object value) {
    ClassLoaderProperties.put(cl, key, value);
  }
}

So the original example now becomes:

Map classLoaders = new WeakHashMap();
ClassLoaderLocal cll = new ClassLoaderLocal();

void addListener(Listener l) {
  ClassLoader cl = l.getClass().getClassLoader();
  List lls = (List) cll.get(cl);
  if (lls == null) {
    lls = new ArrayList();
    cll.set(cl, lls);
  }
  lls.add(l);

  classLoaders.put(cl, Boolean.TRUE);
}

In this code if any listener comes from a freed class loader, then it will be GC-d from both Core.classLoaders and ClassLoaderProperties.classLoaderToHolderClassName, as both are WeakHashMaps and there are no strong references to the class loaders. The generated ClassLoaderLocalMapHolder$$GEN$$X class will also be GC-d along with the class loader, so we have effectively eliminated a class loader leak without explicit cleanup calls from the user.

I hope this code will be useful for someone. I cannot give any guarantees whether it will work or not and it's clearly a hack (though a solid hack). Please use it if you actually understand what is happening. If you see a bug in the code or have a good suggestion, please be sure to comment. There could be a free JavaRebel license in it for you :) Full source code: ClassLoaderLocalMap.java, ClassLoaderLocal.java.

See original post and discussion at dow.ngra.de.