Impala: An Alternative Module Framework

Modular programming is a growing trend in application development.  OSGi is at the center of the modularity movement, however, it is a complex system that requires extra development overhead.  Phil Zoio wanted a less complex module framework that didn't depend on any third-party runtime environments, so he made his own.  He created Impala, a dynamic module framework for Java-based web applications based on the Spring Framework.  DZone recently spoke with Zoio in an exclusive interview about the current state of Impala and the inspiration behind it.

DZone: What is Impala?  What inspired you to create it?

Phil Zoio: I started Impala to address many of the practical problems I encountered when working on a large Spring-based project in around 2006 and 2007. The original developers on the project had attempted to structure the project in a modular way. However, because there wasn't any underlying runtime support for modularity, the attempt was a failure. Without any ability to partially deploy parts of the application in isolation, deploying each change required running a full multi-project build. The result was a slow build, deploy and test cycle, and poor productivity. Also, lack of modularity made it difficult to organize Spring configurations, and resulted in an unnecessarily high level of coupling between projects.

I realized that the solution was not to abandon modularity, but to leverage the technologies already at my disposal - Spring and class loaders - to create some simple mechanisms for supporting modularity at runtime. I considered using OSGi, but did not believe at the time that I would be able to achieve the same level of productivity using OSGi as I have been able to, using the approach I have taken. As of yet, I haven't been given any cause to regret this decision.

The result of subsequent efforts is Impala, a fully dynamic environment for creating and running modular Spring applications. Applications can be broken down in a way which supports and promotes loose coupling, with class loaders enforcing boundaries between modules. These modules can be added, removed and updated on the fly, even without any user intervention. This leads to a very smooth and enjoyable developer experience. Impala also makes it much simpler to write integration tests, because it allows test dependencies to expressed as a collection of modules rather than named Spring configuration files, and allows integration tests to be written in an interactive way.

DZone:  How does Impala fit into the ecosystem of related technologies?
 
Phil:  If you're used to Spring you'll know almost everything you need to know to use Impala. Impala uses Spring to configure the Impala runtime itself - it does not introduce any dependencies that wouldn't be present in the closest equivalent single module Spring application. So the easiest way to think of Impala is providing a simple route to dynamic, modular Spring applications. That being said, in its representation of modules Impala has an architectural layering which would allow for its use with other dependency injection containers.

Impala works for standalone applications as well as web applications running on standard JEE web containers. For web applications, you can of course use Spring MVC, but (with a very small amount of extra effort in some cases) Impala also works other web frameworks, such as Struts (1 and 2), Wicket, Tapestry, JSF, etc.

DZone: How does someone start using Impala?  

Phil:  For day-to-day development on an existing project, you would simply check out a set of Impala projects into an Eclipse workspace, then run up your application or tests. A vanilla Eclipse installation can be used - no additional plugins are required beyond the basic Java tooling.

To kickstart a new project, retrieve third party library dependencies, or build your project, the easiest thing to do right now is to download the Impala distribution, and use Impala's built-in ANT-based build system. This allows you to build a WAR or standalone Jetty-based distribution, run your automated test suites, and perform other tasks.

Support for Maven will be available in the next release milestone, which will available in the next few days.

DZone:  When people think of modularity, they mainly think of OSGi.  Why do you think that is?
 
Phil:  OSGi is a complete from the ground up Java modularity solution which has been around now for around 10 years, so it has had plenty of time to establish itself. The growing importance of modularity in Enterprise Java is a more recent phenomenon. As an established standard widely supported by vendors in the Java community, OSGi was well placed to move into this space as a default modularity choice.

That being said, because of a long separate evolution, bringing OSGi into Enterprise Java has (and continues to be) quite a challenge.

Impala is often positioned as an alternative to OSGi. While this is certainly true, at another level the situation is more complicated. Impala is not fundamentally incompatible with OSGi. There is even a sample of Impala running on OSGi, so full blown support of OSGi as an alternative deployment runtime for Impala applications is a future possibility.

DZone: How does Impala compare to OSGi?  What other technologies can be compared to Impala?  
 
Phil: OSGi on its own is a lower level technology than Impala, so the relevant comparison is really with OSGi in the broader sense, that is, OSGi plus an additional declarative services layer, provided for example by Spring Dynamic Modules.

Impala has evolved to become conceptually quite similar to OSGi, especially in the way that the service registry is used. The big difference is in the way that third party libraries are handled. This is very much part of the scope of OSGi, but not of Impala. Impala is about modularising an application, but not it's use of dependent jars.

The class loader model for Impala is relatively simple, designed to offer a practical solution to common issues. OSGi has a more sophisticated class loader model, capable of addressing pretty much any modularity issue. With Impala, you manage third party libraries in exactly the same way as you would for a regular application (for example, by placing third party libraries in the WEB-INF/lib directory). The advantage is that you also spared the overhead of having to provide fine grained package import and export declarations in your application. Of course, with Impala you don't get versioning and reloading of third party libraries within the application. The big question in view is whether very many applications really require this feature, or use it in anger.



Other differences are more practical than conceptual. Impala applications are easier to package. Testing is also much easier because with OSGi, integration tests need to be OSGi bundles running on an OSGi container. The Impala environment is simpler because of the absence of any third party runtime environment.

DZone: What is the current state of the Impala project?  
 
Phil: The most recent release is 1.0 RC1. Only minor changes are planned between now and the 1.0 final release, planned for early 2010. The most significant of these is to make sure that Impala works well with Maven.

DZone: Give me the project roadmap for Impala?  

Phil: It will definitely be an initial priority to make Impala more easily usable in a wider variety of common existing development environments, and also to make it easier to control and configure. An administration console which can be used to manage module configurations more easily will probably be introduced shortly after the 1.0 release. I'd like to see support for Impala in other IDEs (e.g. NetBeans, IntelliJ). An Eclipse plugin to create Impala projects, interactive with the WTP tooling and use the Impala interactive test runner would be great, too.

Looking ahead, there are also some very exciting but in some cases quite ambitious directions which the project could take in the coming months.

SpringSource are working on a project called Slices which is about making it possible to run typical Java/Spring web applications as separate modules (or slices) within a larger host application. Early versions run only on OSGi, but making this project work outside of OSGi in Impala is a very interesting possibility.
Support for modular Grails applications would be a great additional feature, bringing the modularity benefits of Impala together with the productivity benefits of Grails.

I'd like to add better support for multi-language applications, with different modules built in alternative languages (e.g. Scala or Groovy) according to their strengths, and sharing a common infrastructure.

First class support for OSGi would be another interesting option, which could be done in a number of ways. One is through adding support for Blueprint services specification. Having OSGi support would allow Impala to be used as a more gentle and accessible introduction to modularity, with migration onto OSGi as and when requirements of the project dictate.

There is no precise roadmap for Impala as priorities will depend on demands from users, as well as willingness for others apart from myself to come forward with contributions, so at this point it is impossible to put any precise timescale against any of these features.  Obviously, I would very much appreciate any help members of the developer community are able to provide in helping Impala to reach its potential.
_____
Phil Zoio is an independent developer, architect and consultant, and an active open source developer. In addition to creating Impala, he is the creator of Strecks, as set of Java 5 extensions for Struts. Phil has published articles in numerous leading journals and web sites.  He regularly speaks at Java user groups and conferences.