Frank Kieviet on OpenESB - "Seeing Beyond JCAPS and GlassFish"
The OpenESB project was introduced in 2005 when Sun Microsystems released its initial source code under the CDDL license. While Sun Microsystems, the user community and some other third party companies are working together on the project, the primary source of man power and financial support comes from Sun Microsystems.
The goal of the project is to develop and deliver a high performance, feature rich, modular, JBI compliant ESB runtime as well as development tools to accelerate OpenESB application development.
The runtime uses components from GlassFish application server such as the Metro Web services stack, CLI and administration console infrastructure, and so on. And for the design time OpenESB uses NetBeans IDE as the foundation for providing development and design time artifacts for its runtime components. Design time functionalities are provided in a modular manner on top of NetBean's modularity system.
OpenESB benefits from GlassFish's solid infrastructure to provide a scalable and reliable runtime engine for integrating different software systems based on the JBI standards and its available runtime binding components and design time modules.
DZone recently sat down with OpenESB Technical Lead Frank Kieviet to learn more about the project.
DZone: Hi Frank, can you please introduce yourself and tell us how long you've been working on OpenESB project?
Frank Kieviet: I'm a senior staff engineer at Sun Microsystems. I started working at SeeBeyond in 2002, which at the time was one of the big integration vendors and was acquired by Sun Microsystems in 2005. In the beginning I was responsible as an architect/manager for SeeBeyond's JMS server, and a little later also for SeeBeyond's J2EE server. The scope of my work broadened after the acquisition by Sun to EE-based integration and later also JBI-based integration. In 2004 a small number of people starting working on what would become OpenESB. I have been working on OpenESB off and on since 2006. Last year I became the OpenESB Community Manager. My work is located in Monrovia, which is in Southern California. Can't beat the weather! Can't beat traffic either.
DZone: There is always confusion between OpenESB or so the so-called "GlassFish ESB" and JCAPS. How are they different and when would I use one versus the other?
Frank: I can explain that best by providing a little bit of a historical perspective. Java CAPS was SeeBeyond's Java based Integration product. It was completely based on J2EE. It was also completely closed source and at SeeBeyond we didn't have any plans to open source it. Two things led to OpenESB. First of all, Sun has a different business model and a different culture than SeeBeyond. Open Source is very much a way of life at Sun. *The* way of life, I should say. But Open Sourcing Java CAPS was very difficult because there were many dependencies on commercial non-open source software. Encumbrances is what we call those. Also, Java CAPS didn't lend itself well to an open source style of development: it was not modular enough.
The second thing is that we ran into a number of issues with J2EE, issues that were no longer there in JBI. That is a spec (Java Business Integration) in which it is much easier to create "engines" that have full control over threading, etc. So we decided to do some of the new development based on JBI. This new development was done in open source, and this lead to the OpenESB project. The OpenESB community grew pretty quickly and with the help of the wider community, the number of components grew quickly to more than 40. We incorporated much of OpenESB technology in Java CAPS, but people interested in OpenESB were held back in taking OpenESB into production because of the lack of commercial support for *only* OpenESB: they didn't want to license all of Java CAPS to get only the OpenESB components they were using. Therefore, we decided to provide commercial support on OpenESB too. The problem was that because OpenESB had grown so much, it was difficult to provide support on all components in one go. Instead, we decided to focus on the runtime and a small set of core components first. The runtime is collocated with the GlassFish application server. That's why we called this distribution GlassFish ESB. To make a long story short: OpenESB is the name of the open source project. A commercially supported distribution of the software developed in the OpenESB community is called GlassFish ESB. The same components are also used in Java CAPS, which is a bigger suite of products. This suite has a longer history, and is not open source.
The aim of OpenESB and Java CAPS is to make it easier for developers to build SOA and Integration projects. Most enterprises have dozens if not hundreds of different systems, and at some point all of them need to talk with each other. You typically end up with hundreds of different integration applications. With such a large number of applications, it becomes very beneficial to use a product that provides a reusable solutions for common problems so that you can avoid code duplication. Also, you find a number of typical and difficult problems in integration / SOA applications. We provide out of the box components for those so that developers can avoid reinventing the wheel.
An example of a typical and difficult problem is how you do asynchronous invocations with long running transactions. We have a BPEL engine that provides features such as correlation, compensation handlers, reliability, etc. Correlation solves the problem that you send out a request and get back a response a day later: you need to correlate the response to the original request. This becomes even more difficult if you want to do this in a cluster of machines. Compensation handlers solve the issue you run into when something goes wrong in a long running transaction. You can no longer rollback the transaction: in stead you will have to execute another operation that will undo the original operation.
DZone: JCAPS is the new version of the SeeBeyond ICAN with new features, enhancements, and support plans. Does Sun have any plans to open source more parts of the JCAPS under OpenESB or any other project?
Frank: As not to make things look more complicated, I left out the name ICAN in what I said earlier, but you're right: ICAN is the previous name of Java CAPS. As I mentioned earlier, open sourcing a product that is closed sourced, is very difficult. Just look at the time it took the JDK team to open source Java. Not only are there the encumbrances due to closed source third party components, open sourcing is also much more than just dumping the code in a publicly accessible repository: the build-system, the unit tests, etc. all need to be changed such so that someone new to the software can download it and build it. And that's a pretty big investment. That's why Sun has currently no plans of open sourcing the *complete* Java CAPS suite. Instead, we'll look at parts and components individually and see if it makes sense to open source those. For some it does: last year we did open source a number of components. Some were added to OpenESB, others were added to Mural.
DZone: There are several open source JBI implementations available in the community and almost all of them are supported by commercial entities. How you can position OpenESB and Sun support between Redhat, Progress Software, MuleSource and WSO2?
Frank: There are a few ways of categorizing the competition. First of all, you can look at open standards. OpenESB is built on open standards, most importantly Java EE and JBI. Because of these open standards, we believe that users can avoid vendor lock-in because components are portable between different vendors. Most of the other projects you just mentioned are not built on standards, but instead have invented their own proprietary standards.
Secondly, you can look at the reach into the enterprise. Sun Microsystems is backing OpenESB, and that gives many enterprise customers the reassurance that the project will be supported for years to come and that commercial support by a reputable company is and will remain available. Of the companies you just mentioned by the way, one already seized to exist as an independent company. And it's not just about size and reputation, it's also about the breadth of product offering: unlike companies that only provide an integration product, Sun is offering much more than just GlassFish ESB: it's providing the full stack including the operating system, middleware and database.
Thirdly, you can look at open source. Missing from your list are companies such as IBM and Oracle. We encounter them often in sales deals -- sometimes we loose to them, but often we win from them. What our product has in common with them is the backing and continuity by a large and established company and the breadth of product offering. The difference however is in open source: OpenESB is completely open source.
When you compare products on these three factors, you see that OpenESB is
DZone: Can you tell us a little more about the Mural community and its projects?
Frank: Yes, where OpenESB focuses on connectivity, data transformation and data/message flow, Mural focuses on a different data aspects, e.g. how to get a single view on data entities.
DZone: What is the ESB Console? When can we expect to see a stable release for this?
Frank: Connectivity, data transformation and data flow are just one aspect of running an integration or SOA application. We're trying to add a lot of value to what happens after deployment. The ESB Console provides management and monitoring capabilities for the applications after they're deployed. This becomes more complicated when you're running an application on more than just one box. That's the kind of work we're trying to do in the ESB Console. As you can see on our roadmap, we're focusing on getting the 2.1 and v3 release out first. The ESB Console is not planned as a supported component in these releases yet. The roadmap beyond that has not been agreed on, so it would be premature for me to say when it will be supported. Let me just say that there's a lot of interest in the ESB Console.
The ESB console project is available at http://wiki.open-esb.java.net/Wiki.jsp?page=ESBConsoleProject
DZone: The Fuji project has been under development for quite some time now. What's the status of this project?
Frank: Project Fuji was started as a research project last year. The idea was to make sure that we're not just focusing on incremental changes, but also keeping an eye open for fundamental improvements that we can make to the platform. Fundamental improvements typically require drastic changes and more out-of-the-box thinking, so that's not something you can do in an incremental release. We had a few people working on Fuji, and they tried a number of revolutionary ideas. Last week we decided which of these ideas will be part of our upcoming V3 release. So now we're decreasing the extent of experimentation, and are working on productization with increased focus.
What this brings to the V3 release is Maven support, OSGi, interceptors, a number of built-in Enterprise Integration Patterns, and a domain specific language to express how messages should flow through the system.
DZone: OpenESB has many binding components and service engines; do you have a rough estimation of the number of contributed components versus the number of components developed by Sun?
Frank: Of the 45 components, 11 components were created by 5 organizations other than Sun Microsystems. I'm expecting to announce another component next week by the way, which again is built by a contributor other than Sun. I should also add that we're expecting that another handful of components are going to be further developed and maintained by partners. I'm very happy with the broad participation that we're seeing in the community.
A list of all components is available at https://open-esb.dev.java.net/Components.html
DZone: Can you give us some advice on your ESB products in general, some pitfalls and best practices which you think are most important items?
Frank: Next to what I mentioned earlier about open source, open standards, and support, I should add that when you're comparing multiple ESB products, look at how they stack up against each other when you compare usability and productivity.
Most integration or SOA projects are staffed with several dozen different people with different skill sets who together build sometimes hundreds of different projects. Will you end up with a swamp of hundreds of XML files that you need to edit by hand, or will you be able to use tooling that will give you a more graphical overview of what does what? Does the ESB promote and facilitate reuse? Those are important questions to consider.
DZone: How scalable is OpenESB? Is there a clustering model behind it, specifically for BPEL engines?
Frank: There are two models for clustering: heterogeneously and homogeneously. In OpenESB we're currently providing homogeneous clustering, meaning that a set of machines have identical OpenESB configurations and an external load balancer distributes load between those machines. We're supporting this model using the clustering technology that's available in GlassFish.
In heterogeneous clustering, nodes can distribute load between themselves and nodes can have different configurations. An example of a simple heterogeneous implementation is based on JMS queues to distribute load. We support this model, but what we're trying to do beyond that is to avoid the overhead of JMS, but still provide the advantages that JMS provides, e.g. location transparency / discovery etc. This is a model that we're trying to support through the proxy binding component, which is a component that we're targeting for a future release.
Irrespective of the clustering model, the biggest problem to solve is that of stateful asynchronous interactions. In web applications you would be talking about servlet session objects. In SOA you would typically be talking about long running processes in BPEL. Our BPEL engine supports distribution for any of the models I mentioned earlier. It does this by using a shared database. The BPEL engines that share this database take care of figuring out which engine is "owning" a business process, and if a response comes in on a different engine, the business process will migrate automatically to the engine on which the response came in.
DZone: Does Sun Microsystems have any plans to provide a BPM suite based on its currently available products?
Frank: BPM is a term that means different things to different people. We're currently supporting a form of BPM, and we'll be extending that functionality further in the future. On the other hand, we don't intend to go as far as the BPM pure-players. So BPM will remain part of what we're currently offering, and as far as I know there's no plan for a separate BPM product suite.
DZone: Thanks for your time today Frank. Any final words for the DZone community?
Frank: If you're interested in OpenESB, please visit our web site. We have a video there that will give you an impression of how OpenESB looks like. Next, download OpenESB and give it a try. You'll find the mailing list pretty responsive and friendly to your questions and comments. I hope to see you there!