Platinum Partner
java,jboss,architecture,microcontainer,classloading,virtual file system

A Look Inside JBoss Microcontainer's ClassLoading Layer

Now that we've grasped the Virtual File System (VFS) concepts, it’s time we move on to Microcontainer’s ClassLoading layer (as promised in the previous article).

JBoss has always had a unique way of dealing with classloading, and the new classloading layer that comes with Microcontainer is no exception (keep in mind that you can use Microcontainer without classloading layer - it's an add-on that you use when you want non-default classloading). With OSGi-style classloading getting more and more traction, and a number of new Java modules/classloading specifications on the horizon, it was high time we revamped our classloading layer in order to be able to easily and nicely support the new demands.

Read the other parts in DZone's exclusive JBoss Microcontainer Series:

 

Learning from our previous mistakes, we wanted to provide a flexible architecture, while at the same time limit exposure of nitty-gritty details, thereby minimizing potential 'hack' uses of  implementation-detail classes - the ones not part of a public API.

As you will see, most if not all of the details are hidden behind private and package private methods, without compromising the extensibility and functionality available through public classes and methods that make the API. This, in terms of new classloading layer, means that you code against policy and not against classloader details.

The ClassLoader project is split into 3 subprojects

  • classloader
  • classloading
  • classloading-vfs

“classloader” contains our custom java.lang.ClassLoader extension without any specific classloading policy - knowledge of where to load from and how to load.

“classloading” is an extension of Microcontainer’s dependency mechanisms, based on classloading configuration, and “classloading-vfs” as its VFS backed implementation - VFS is used to load classes.

ClassLoader

The purpose of the “classloader” subproject is to provide the actual runtime classloading mechanism. It provides the ClassLoader implementation, that supports pluggable policies and is itself pretty much a final class - not to be tempered with.

Instead, people that want to write their own ClassLoader implementations will write a ClassLoaderPolicy which provides a simpler api for locating classes and resources, and for specifying other rules associated with the classloader.

Users who want to tinker with classloading instantiate a ClassLoaderPolicy and register it with a ClassLoaderSystem to create a custom ClassLoader. They may also create a ClassLoaderDomain to partition the ClassLoaderSystem.

This layer also includes the implementation of things like DelegateLoader model, classloading, resource filters, and parent-child delegation policies.

The runtime is JMX enabled to expose the policy used for each classloader, along with classloading statistics, and debugging methods to help determine where things are loaded from.

 

 

 

As most of us know, classloading can be a real PITA. Fortunately, we hide most of the nasty implementation details. Let’s have a look then at the one piece that makes your classloading run the way you want it – the ClassLoaderPolicy.

public abstract class ClassLoaderPolicy extends BaseClassLoaderPolicy
{
public DelegateLoader getExported()

public String[] getPackageNames()

protected List<? extends DelegateLoader> getDelegates()

protected boolean isImportAll()
protected boolean isCacheable()
protected boolean isBlackListable()

public abstract URL getResource(String path);

public InputStream getResourceAsStream(String path)

public abstract void getResources(String name, Set<URL> urls) throws IOException;

protected ProtectionDomain getProtectionDomain(String className, String path)
public PackageInformation getPackageInformation(String packageName)
public PackageInformation getClassPackageInformation(String className, String packageName)

protected ClassLoader isJDKRequest(String name)
}

 

Let’s look at two examples of ClassLoaderPolicy.

The first one knows how to retrieve resources based on regular expression, while the second one knows how to handle encrypted resources.

Both examples use the code from Microcontainer demos module called 'policy'. The main class is ClassLoaderMain from demos module called 'classloader'.

(You can find all the demos in jboss svn repository)

jboss-demos-policy.jar contents:

config/excluded.properties
config/properties.xml
config/settings.txt
META-INF/encrypted-beans.xml
META-INF/regexp-beans.xml
org/jboss/demos/policy/services/EncryptedService.class
org/jboss/demos/policy/services/PrintService.class
org/jboss/demos/policy/services/RegexpService.class

 

You can see here, that we have two services (*-beans.xml in META-INF), which access resources (in config/ directory) via classloader.

public class RegexpClassLoaderPolicy extends ClassLoaderPolicy
{
private VirtualFile[] roots;
private String[] packageNames;

public RegexpClassLoaderPolicy(VirtualFile[] roots)
{
this.roots = roots;
}

@Override
public String[] getPackageNames()
{
if (packageNames == null)
{
Set<String> exportedPackages = PackageVisitor.determineAllPackages(roots, null, ExportAll.NON_EMPTY, null, null, null);
packageNames = exportedPackages.toArray(new String[exportedPackages.size()]);
}
return packageNames;
}

protected Pattern createPattern(String regexp)
{
boolean outside = true;
StringBuilder builder = new StringBuilder();
for (int i = 0; i < regexp.length(); i++)
{
char ch = regexp.charAt(i);
if ((ch == '[' || ch == ']' || ch == '.') && escaped(regexp, i) == false)
{
switch (ch)
{
case '[' : outside = false; break;
case ']' : outside = true; break;
case '.' : if (outside) builder.append("\\"); break;
}
}

builder.append(ch);
}
return Pattern.compile(builder.toString());
}

protected boolean escaped(String regexp, int i)
{
return i > 0 && regexp.charAt(i - 1) == '\\';
}

public URL getResource(String path)
{
Pattern pattern = createPattern(path);
for (VirtualFile root : roots)
{
URL url = findURL(root, root, pattern);
if (url != null)
return url;
}
return null;
}

private URL findURL(VirtualFile root, VirtualFile file, Pattern pattern)
{
try
{
String path = AbstractStructureDeployer.getRelativePath(root, file);
Matcher matcher = pattern.matcher(path);
if (matcher.matches())
return file.toURL();

List<VirtualFile> children = file.getChildren();
for (VirtualFile child : children)
{
URL url = findURL(root, child, pattern);
if (url != null)
return url;
}

return null;
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}

public void getResources(String name, Set<URL> urls) throws IOException
{
Pattern pattern = createPattern(name);
for (VirtualFile root : roots)
{
RegexpVisitor visitor = new RegexpVisitor(root, pattern);
root.visit(visitor);
urls.addAll(visitor.getUrls());
}
}

private static class RegexpVisitor implements VirtualFileVisitor
{
private VirtualFile root;
private Pattern pattern;
private Set<URL> urls = new HashSet<URL>();

private RegexpVisitor(VirtualFile root, Pattern pattern)
{
this.root = root;
this.pattern = pattern;
}

public VisitorAttributes getAttributes()
{
return VisitorAttributes.RECURSE_LEAVES_ONLY;
}

public void visit(VirtualFile file)
{
try
{
String path = AbstractStructureDeployer.getRelativePath(root, file);
Matcher matcher = pattern.matcher(path);
if (matcher.matches())
urls.add(file.toURL());
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}

public Set<URL> getUrls()
{
return urls;
}
}
}

 

RegexpClassLoaderPolicy uses a pretty naïve mechanism to find matching resources. Real-life implementation could analyze regular expression better, and try to avoid visiting all resources while looking for a matching one.

public class RegexpService extends PrintService
{
public void start() throws Exception
{
System.out.println();

ClassLoader cl = getClass().getClassLoader();
Enumeration<URL> urls = cl.getResources("config/[^.]+\\.[^.]{1,4}");
while (urls.hasMoreElements())
{
URL url = urls.nextElement();
print(url.openStream(), url.toExternalForm());
}
}
}

 

Regexp service uses "config/[^.]+\\.[^.]{1,4}" regular expression to list resources under config/ directory. We limit the suffix length, therefore we expect excluded.properties file to not be listed.

public class CrypterClassLoaderPolicy extends VFSClassLoaderPolicy
{
private Crypter crypter;

public CrypterClassLoaderPolicy(String name, VirtualFile[] roots, VirtualFile[] excludedRoots, Crypter crypter)
{
super(name, roots, excludedRoots);
this.crypter = crypter;
}

@Override
public URL getResource(String path)
{
try
{
URL resource = super.getResource(path);
return wrap(resource);
}
catch (IOException e)
{
throw new RuntimeException(e);
}
}

@Override
public InputStream getResourceAsStream(String path)
{
InputStream stream = super.getResourceAsStream(path);
return crypter.crypt(stream);
}

@Override
public void getResources(String name, Set<URL> urls) throws IOException
{
super.getResources(name, urls);
Set<URL> temp = new HashSet<URL>(urls.size());
for (URL url : urls)
{
temp.add(wrap(url));
}
urls.clear();
urls.addAll(temp);
}

protected URL wrap(URL url) throws IOException
{
return new URL(url.getProtocol(), url.getHost(), url.getPort(), url.getFile(), new CrypterURLStreamHandler(crypter));
}
}

 

In this example we provide a simple tool (Tools class) to encrypt our jars. By specifying a proper filter, we can configure which resources we want to encrypt. In our case we encrypt everything except META-INF contents. For encryption we use JDK’s Cipher class.

public class EncryptedService extends PrintService
{
public void start() throws Exception
{
ClassLoader cl = getClass().getClassLoader();

URL url = cl.getResource("config/settings.txt");
if (url == null)
throw new IllegalArgumentException("No such settings.txt.");

InputStream is = url.openStream();
print(is, "Printing settings:\n");

is = cl.getResourceAsStream("config/properties.xml");
if (is == null)
throw new IllegalArgumentException("No such properties.xml.");

print(is, "\nPrinting properties:\n");
}
}

 

This service actually doesn’t do much. It just prints out the contents of two configuration files. The point here is to show that decryption of any encrypted resources is hidden behind the classloading layer.

To properly test this, you should either encrypt the policy module yourself or use an existing encrypted one: see ${demos.home}/policy/src/etc/crypto-([0-9]+).jar file

In order to put this into action, we only need to properly tie EncryptedService to ClassLoaderSystem, and deployers. To see how this is done, check the demo code.

We’ll talk more about partitioning ClassLoaderSystem in the following section.

ClassLoading

The ClassLoading subproject represents a higher level abstraction whose main role is to control classloading dependencies.

Users at this level, instead of using the ClassLoader abstraction directly, will create ClassLoading Modules which contain declarations of ClassLoader dependencies. Once the dependencies are specified the ClassLoader(Policy)s are constructed and wired together accordingly.

To facilitate defining the ClassLoaders before they actually exist, the abstraction includes a ClassLoadingMetaData model.

The ClassLoadingMetaData can be exposed as a "Managed Object" within the new JBoss5 profile service, allowing system admins to deal with higher level policy details rather than the implementation details.

 


A simple diagram of how "classloading" mechanism looks like.

public class ClassLoadingMetaData extends NameAndVersionSupport
{
/** The serialVersionUID */
private static final long serialVersionUID = -2782951093046585620L;

/** The classloading domain */
private String domain;

/** The parent domain */
private String parentDomain;

/** Whether to make a subdeployment classloader a top-level classloader */
private boolean topLevelClassLoader = false;

/** Whether to enforce j2se classloading compliance */
private boolean j2seClassLoadingCompliance = true;

/** Whether we are cacheable */
private boolean cacheable = true;

/** Whether we are blacklistable */
private boolean blackListable = true;

/** Whether to export all */
private ExportAll exportAll;

/** Whether to import all */
private boolean importAll;

/** The included packages */
private String includedPackages;

/** The excluded packages */
private String excludedPackages;

/** The excluded for export */
private String excludedExportPackages;

/** The included packages */
private ClassFilter included;

/** The excluded packages */
private ClassFilter excluded;

/** The excluded for export */
private ClassFilter excludedExport;

/** The requirements */
private RequirementsMetaData requirements = new RequirementsMetaData();

/** The capabilities */
private CapabilitiesMetaData capabilities = new CapabilitiesMetaData();

... setters & getters

ClassLoadingMetaData api:

<classloading xmlns="urn:jboss:classloading:1.0"
name="ptd-jsf-1.0.war"
domain="ptd-jsf-1.0.war"
parent-domain="ptd-ear-1.0.ear"
export-all="NON_EMPTY"
import-all="true"
parent-first="true"/>

 

This is an example of how ClassLoadingMetaData looks via xml.

      ClassLoadingMetaData clmd = new ClassLoadingMetaData();
if (name != null)
clmd.setDomain(name + "_Domain");
clmd.setParentDomain(parentDomain);
clmd.setImportAll(true);
clmd.setExportAll(ExportAll.NON_EMPTY);
clmd.setVersion(Version.DEFAULT_VERSION);

 

Or how it looks when you hack it in code.

You can add ClassLoadingMetaData to your deployment either programmatically, or declaratively - via jboss-classloading.xml.

We’ll use the xml approach to define different examples / configurations.

Let’s look then at a few examples of ClassLoadingMetaData, that will cover some of the most common use-cases. These examples should cover enough for you to be able to easily handle your custom classloading needs.

<classloading xmlns="urn:jboss:classloading:1.0"
domain="DefaultDomain"
top-level-classloader="true"
export-all="NON_EMPTY"
import-all="true">
</classloading>

 

First, let's show an example of a legacy configuration, which we can call 'big-ball-o-mud' (TM by Adrian Brock :-). Here we put the classloader in the "DefaultDomain" which is shared amongst all the applications that don't define their own domain:

<classloading xmlns="urn:jboss:classloading:1.0"
domain="IsolatedDomain"
export-all="NON_EMPTY"
import-all="true">
</classloading>

 

This is your typical run-of-the-mill enterprise classloading isolation:

<classloading xmlns="urn:jboss:classloading:1.0"
domain="IsolatedWithParentDomain"
parent-domain="DefaultDomain"
export-all="NON_EMPTY"
import-all="true">
</classloading>

 

More isolation, this time with an explicit parent:

<classloading xmlns="urn:jboss:classloading:1.0"
parent-first="false">
</classloading>

 

This is what we call non-j2seClassLoadingCompliance, something .war deployments do by default.
Instead of doing default parent-first lookup, you first check your own resources:

<classloading xmlns="urn:jboss:classloading:1.0">
<requirements>
<package name="org.jboss.dependency.spi"/>
</requirements>
<capabilities>
<package name="org.jboss.cache.api"/>
<package name="org.jboss.kernel.spi"/>
</capabilities>
</classloading>

 

An OSGi kind of configuration:

<classloading xmlns="urn:jboss:classloading:1.0">
<requirements>
<module name="jboss-reflect.jar"/>
</requirements>
<capabilities>
<module name="jboss-cache.jar"/>
</capabilities>
</classloading> 

 

Instead of fine grained packages, we can import/export whole modules/libraries. 

<classloading xmlns="urn:jboss:classloading:1.0">
<requirements>
<package name="si.acme.foobar"/>
<module name="jboss-reflect.jar"/>
</requirements>
<capabilities>
<package name="org.alesj.cl"/>
<module name="jboss-cache.jar"/>
</capabilities>
</classloading> 

 

We can of course also mix the requirements and capabilities types; using packages and/or modules. 

Another useful feature in “classloading” sub-project is its elegantly small resource-visitor-pattern implementation.

In ClassLoader project the connection between deployment and classloading is done through Module class. It is this class that holds all of the required information to properly apply restrictions on the visitor pattern (e.g. filtering):

public interface ResourceVisitor
{
ResourceFilter getFilter();

void visit(ResourceContext resource);
}

public interface ResourceContext
{
URL getUrl();

ClassLoader getClassLoader();

String getResourceName();

String getClassName();

boolean isClass();

Class<?> loadClass();

InputStream getInputStream() throws IOException;

byte[] getBytes() throws IOException;
}

The usage is very simple, just instantiate your ResourceVisitor instance and pass it to Module::visit method.

In our deployment framework we use this feature to index annotations usage in deployments.

ClassLoading VFS

The “classloading-vfs” provides an implementation of the ClassLoaderPolicy that uses JBoss Virtual File System project for loading classes and resources. This can be used directly or in combination with “classloading” framework.

Optionally, you can set up your modules inside Microcontainer configuration.

<deployment xmlns="urn:jboss:bean-deployer:2.0">

<classloader name="anys-classloader" xmlns="urn:jboss:classloader:1.0" import-all="true" domain="Anys" parent-domain="DefaultDomain">
<capabilities>
<package name="org.jboss.test.deployers.vfs.reflect.support.web"/>
</capabilities>
<root>${jboss.tests.url}</root>
</classloader>

<bean name="AnyServlet" class="org.jboss.test.deployers.vfs.reflect.support.web.AnyServlet">
<classloader><inject bean="anys-classloader:0.0.0"/></classloader>
</bean>

</deployment>

 

This xml code is transformed (via VFSClassLoaderFactory) into VFSClassLoaderPolicyModule which then creates the actual ClassLoader instance.

You can then directly use this new ClassLoader instance with your beans.

Note: VFSClassLoaderFactory extends ClassLoadingMetaData, so all previous configuration examples apply in this case as well.

We’ve reached the end of this long JBoss ClassLoader introduction. I hope it did shed some light on this otherwise complex topic, making your life a bit easier when dealing with another ClassNotFoundException issue within your JBoss environment.

While we have already used Deployers heavily in JBoss Microcontainer articles, we haven't properly explained their usage and concepts yet. That will be the topic of our next article. Stay tuned!

P.S.: Thanks to Adrian for letting me use some of his documentation work for this article. And Marko for doing the editing of this article.

 

About the Author

Ales Justin was born in Ljubljana, Slovenia and graduated with a degree in mathematics from the University of Ljubljana. He fell in love with Java seven years ago and has spent most of his time developing information systems, ranging from customer service to energy management. He joined JBoss in 2006 to work full time on the Microcontainer project, currently serving as its lead. He also contributes to JBoss AS and is Seam and Spring integration specialist. He represent JBoss on 'JSR-291 Dynamic Component Support for Java SE' and 'OSGi' expert groups.

 

{{ tag }}, {{tag}},

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}
{{ parent.authors[0].realName || parent.author}}

{{ parent.authors[0].tagline || parent.tagline }}

{{ parent.views }} ViewsClicks
Tweet

{{parent.nComments}}