Using OSGi & Equinox Effectively: An Interview With Jeff McAffer
After releasing his latest book covering OSGi & Equinox, I took the opportunity to catch up with Jeff McAffer. In this interview we discuss the basics of OSGi, the Equinox implementation, benefits of modularity and some best practices around using Equinox.
James: Can you introduce yourself and your associations with Eclipse?
Jeff: I'm Jeff McAffer, the CTO and co-founder of EclipseSource. I've been involved in Eclipse since developing its first prototypes while working at IBM more than 10 years ago. Since that time I've focused mainly on runtime work and have led various efforts from the Runtime to Equinox to the evolution of Eclipse as a Rich Client Platform and now the EclipseRT effort to drive Eclipse runtime technologies. Along the way I've been on the Board of Directors, several PMCs and the Architecture Council.
James: What audiences are you targetting with the book - it's more than just the Eclipse crowd?
Jeff: Oh yes. The title is OSGi and Equinox -- that is, OSGi with some detail on Equinox. As you know OSGi is at the heart of Eclipse but the technology is applicable in a much broader scope. We see OSGi being used in embedded environments (its original target audience), desktop apps (with Eclipse RCP and other application infrastructure), a wide range of server side infrastructure and most of the major Java application servers. Many people in the enterprise Java space are look at and talking about OSGi. This book addresses all of these readers. It seeks to make clear the basic and some advanced topics in a generic OSGi way. Of course we use Equinox as the example framework and Eclipse for tooling but the vast majority of the material and samples apply to any OSGi system.
James: Could you give a really simple explanation of OSGi, and then what Equinox is?
Jeff: At its simplest, OSGi is the specification for a module runtime for Java. Equinox is the reference implementation for the core OSGi of that specification. Of course, both are quite a bit more than that. OSGi has many add-on services from the embedded to enterprise domains. Equinox covers many topics that are not dealt with in the specs (e.g., provisioning, native launching, non-OSGi-based code library integration, dynamic aspect weaving, ...).
James: Do you see Equinox & OSGi as the core for any modular system, or can it be overkill in some cases?
Jeff: OSGi is elegantly minimal -- just 27 Java types are defined in the core specification. If you are building a Java application and are interested in what modularity has to offer, I don't see any viable alternative to OSGi. Implementations such as Concierge can be quite small (e.g., <100k). Other implementations are larger but more performant or more fully featured. As always, use the right runtime for the job.
James: What are the key benefits of OSGi?
Jeff: OSGi is a simple but powerful modularity framework for Java. Using OSGi means that you can write self-contained modules with clear API boundaries and strong runtime enforcement of those boundaries. This reduces the amount of code you have to write to create your system because you can reuse other people's off-the-shelf modules. This sounds great to geeks but the appeal is wider than that.
As we say in Chapter 1 of the book, "Modularity ... is the lubricant of collaboration." Modularity enables freedom of action for both producers and consumers of technology. This fundamentally changes the way you develope and deploy software solutions at both a personal and organizational level. The NASA guys are a great example of this. They have vastly diverse teams of people collaborating in space missions. OSGi, initially in the form of Eclipse RCP but now on the server as well, empowered all members of the team participate in creating the solution. In some ways the power of OSGi extends beyond systems engineering and into social engineering.
James: What has put Equinox ahead of other OSGi implementations?
Jeff: I would not say "ahead" but rather "different". There are a handful of open source OSGi implementations and each has focused on a different set of users. Equinox has benefitted from years of widespread use as the runtime for the Eclipse IDE and RCP-based applications. Each day, literally millions of people are running Equinox-based systems on their desktop. The requirements of those applications have pressed the Equinox team to drive scaleability and robustness as well as paying attention to edge and corner cases that are part of real software systems. In the last few years Equinox has come to be the framework-of-choice for server implementors from Websphere to Spring dm to SAP NetWeaver. This has further enhanced the state of Equinox as an industrial-strength OSGi framework.
James: How difficult was it to write the book? Did it take long?
Jeff: This is the second book project in which I have been involved. This go around was a little more drawn out (it took about 1.5 years) but was actually easier. That is partially because of lessons learned but also because we started from some existing material. We were able to seed a few chapters from the RCP book, Toast, the sample application we develop through out the book, was largely already implemented, we could reuse the Samples Manager to manage the code, and many of the production processes were well understood. All in all it was also a huge amount of fun. Paul and Simon are awesome to work with and Toast is very motivating. Part of the challenge was stopping from adding more cool stuff to Toast!
Probably the biggest challenge was hitting the different levels of detail for the different readers. We wanted to create a book that would ease people into OSGi but then not leave them hanging. So many tutorials make everything look easy but when you try for yourself there are many unanswered questions. OSGi is actually very simple on the surface--create a bundle, describe its dependencies, drop it in a system and shazaam. That is, until you build a real system. Then all the gorpy details come to play--third-party non-OSGi code, API definition and evolution, dynamic behaviour, and the like. In the book we give readers the fundamental understanding they need to craft their own systems. Envisioning and creating that content was hard work.
James: What is the key tip that you would give to anyone thinking about, or already, using OSGi?
Jeff: Good fences make good neighbors. I love my house but we have eight direct neighbors. I've got hours of stories involving police, ambulances, building inspectors, heavy equipment, lawyers, courts, you name it. On reflection, the vast majority of those stories revolve around ill-defined boundaries and behaviours. And so it is with software systems. OSGi gives you the tools you need to define the boundaries between elements of the system--modules. It then enforces those boundaries at runtime. So the key tip is to pay attention to your API definition and evolution--maintain your fences.
James: What is the trickiest part to OSGi?
Jeff: The things that frustrate people the most about OSGi are not OSGi at all but what it enables--modularity and dynamism. Defining a module requires you to think about the boundaries and behaviours you want the module to have. Many systems are only loosely defined and evolve quickly. This in turn requires you to think harder about your modules and their interaction and refactor frequently. OSGi enables you to capture and enforce these interactions. Of course, you can still write big wads of spaghetti code if you like but that is a choice independent of OSGi. Similarly, if you want to have a system where function can come and go spontaneously, you have to account for that. Again, OSGi enables the creation of such systems but, well, there's no free lunch, you still have to code accordingly.
Of course, there are some quirks in the way OSGi works and the impedance mismatch with other extensibility mechanisms like context classloaders and the like cause people no end of grief. For the most part these issues are dealt with once and progress continues.
James: It seems to me that following your examples, you get some best practices for programming in general. Could you list out some of your best practices for working with OSGi applications?
Jeff: Don't program OSGi -- In my OSGi talks and training courses I like to start off saying "The best practice for programming OSGi is to not use OSGi". What I mean by that is, write POJOs and use dependency injection style. Too many people have code that reaches all over the place and has OSGi API calls littered throughout. For the most part this makes the code hard to read, hard to test and difficult to reuse in other contexts. Full up Toast has about 80 bundles and only a handful reference OSGi core framework APIs!
Use Declarative Services -- Go ahead and put down the ServiceTracker rocks and sticks and make the move to Declarative Services. DS is a key part to eliminating the widespread use of OSGi APIs I mentioned earlier. It greatly simplifies the collaboration between bundles using OSGi services by eliminating ALL of the boilerplate and brittle/complicated code otherwise needed.
Understand the bundle food chain -- Put another way, minimize and manage your dependencies. Be realistic about the bundles you are using and why. I've never been a fan of duplication but have also seen cases where many megabytes of code was sucked in just to get a simple constant!
Don't use Bundle activators -- Sure, sometimes the function in a bundle needs to be setup but this should be the exception. This is especially true if you are using DS. The downside of activators is that they take time on startup and they generally capture some sort of singleton-like behaviour. For example, the typical HTTPService bundle has an activator that starts the HTTP server listening on a configured port. What if you want two? And want them started at different times? And, ...
James: What is the most common mistake you see with OSGi use?
Jeff: Based on the best practices I mentioned earlier, the biggest problem I see is people using too much OSGi API too often, especially around services. Seriously, look in your bundle manifests for "org.osgi" and justify each occurrence. Likely most cases are where you are reaching out to the rest of the world. That forces you to make assumptions about your surroundings and, as a result, limits the situations in which your bundle can be used. Use DS or some other injection mechanism to have these things injected into your bundle.
James: What's your opinion on modularity: design it in up front, or refactor it in as you go?
Jeff: Great question! With just about everything it is better if you design and implement for your actual requirements. I cringe at the thought that not all teams have the benefits of modularity at the top of their priority list, but hey, that's reality. If modularity is something that factors into your thinking, then absolutely, design it in as soon as possible. I've seen some pretty major modularity retrofitting projects and they are a struggle. In Eclipse itself we have done some of these and even with our inherent modularity, it can be a struggle. By far the most grief came from places where things were tightly coupled and a jumble of different concerns (i.e., not cohesive).
Actually, in the book we talk about this and point to high cohesion and loose coupling explicitly as best programming practices. With or without explicit modularity, if you do these well, your software is more modular and the approach brings clarity to the design process. It also makes refactoring easier (and less necessary). So, spending time designing and coding for loose coupling is a little more effort up front but has huge payoffs in the medium to long term.
In the end, reality is that requirements are unclear and changing. You have to be ready, willing and able to refactor. Then refactor more. And do it again. Getting modularity in earlier helps with that but good design processes are more important.
James: Where is Equinox going in the future? Does it remain at the core of the e4 effort?
Jeff: Equinox, the framework, will continue to evolve with the OSGi specifications and be a powerful and robust implementation. It will take cues from projects like e4, end users and real-world usecases, and evolve to meet those needs. The Equinox project itself is expanding to address broader modularity needs. Last year we added dynamic code weaving capabilities to the release. This year we are excited to see the ScalaModules work moving to Equinox. In addition, we have been enhancing the server side and provisioning efforts in Equinox.
The real story however is bigger than just Equinox, it is the community and ecosystem that it enables. A couple years ago at Eclipse we kicked off the EclipseRT effort to drive the use of Eclipse technology in the runtime space. Since then EclipseRT has grown to include many well-known runtime technologies such as EclipseLink (formerly TopLink from Oracle) and Jetty. The Gemini and Virgo projects were recently announced and represent vast swaths of Oracle and SpringSource runtime technology being contributed to EclipseRT. In particular, the Spring dm Server is moving to Eclipse as part of Virgo. So Equinox and EclipseRT have a huge and very exciting future.
James: How does p2 fit in to all this? And what alternatives are their to provisioning?
Jeff: p2 is part of Equinox and is the latest generation provisioning technology from Eclipse. While it is designed as a general provisioning infrastructure, it excels at managing OSGi-based systems big and small. One of the reasons p2 exists is that everyone's provisioning problem is different. It is extremely flexible, extensible and configurable. The book covers p2 in considerable detail and the code samples show how to integrate p2 at a deep level in support of dynamic and remote provisioning.
Of course, wherever there is diverse need, there is diverse supply. There are a number of alternatives to p2. They all have their strengths and weaknesses so teams should pick the one that best suits their needs. The OSGi specifications have a couple related services in Initial Provisioning and Deployment Admin, Apache Ace is relatively new on the scene, there are commercial options from traditional OSGi vendors. Perhaps the most pervasive though is home-grown solutions. It is amazing how inventive people get in the provisioning space.
James: Since you started working for Eclipse almost a decade ago, what have been the most exciting changes you've seen?
Jeff: Now that you mention it, I've been working on Eclipse since early 1999. Wow, that's a long time! Over that time I've seen three major transitions. The first was from an unknown, proprietary, niche effort to an open source phenomenon and domination of the tooling business. That was exciting and gratifying. I learned a lot about the technology, modularity, community and open source. The next transition was the evolution of Eclipse as a Rich Client Platform. Very cool. This was a community driven affair with people taking our work and using it in ways that we never dreamed of. To me, that validated the whole concept behind modularity. The third transition, we are witnessing now--Eclipse in the runtime or EclipseRT. The rate and pace of adoption in the runtime space is staggering. Every day someone comes out of the woodwork looking to use or contribute something new. We ran a series of EclipseRT days last fall and the range of attendees and was they were doing was very exciting. I don't know what we'll do to top this. Guess we'll find out in a year or two...