Modular Farm Management on the NetBeans Platform

I'm Timon Veenstra and I live in Groningen, in the Netherlands. Currently, I am working at Ordina as a software engineer and technical architect.

Below follows an overview of "AgroSense", which is a rich-client platform providing a highly modular farm management system on the NetBeans Platform:

Overview of "AgroSense"

Agriculture is rapidly becoming a very data intensive industry. A multitude of agricultural specialities is needed to transform all this data into useful farm management information, which can lead to higher productivity, less use of pesticides, etc. Specific agricultural knowledge is often owned by small companies, not capable of developing an overall competitive farm management system.

AgroSense provides an open source modular framework in which modules from several sources are combined into one integrated farm management system. Modules can be provided by governmental departments, universities, or commercial businesses. Some will be free of charge, while others will require some form of payment. In the end, the
farmer decides which modules contribute to the business:

History of the Application

This was my first rich-client application based on Maven. We had created some Ant-based NetBeans Platform applications previously. At first, it seemed the Ant library wrappers were no longer needed. Adding things we needed as dependencies of the module worked fine, until we created our second module and started using the API of the first. It took us several days of gremlin hunting before we concluded that we still needed the module wrappers. After adding those to our module architecture and sticking with the designed dependencies, we solved that problem.

The second problem to solve was our persistence layer. We wanted to support multiple persistence units, but still stay in control of our database. We solved this puzzle by creating three database modules, plus a Hibernate/hsqldb library wrapper.

The database API module contains an interface describing how modules should make themselves known, i.e., as wanting to do something with persistence (persistence unit name, etc.), and an interface to obtain an "EntityManager". The second module contains our test database implementation, it provides an "EntityManager" connection to an in-memory Derby database and assumes a persistence unit named "test". The third module contains our local database.

The local database "EntityManagerProvider" creates an "EntityManager" factory for each module that provides persistence information through the service provider mechanism, i.e., implementations of the "PersistingModule" interface in the database API module. This results in a connection per persistence unit. Since Derby allows only one connection to its database, we had to switch to hsqldb instead.

With a little utility class we managed to make Hibernate play along as well:

public class JavaAssistHelper {
    public static ClassLoaderProvider createJavaAssistClassLoader() {
        ProxyFactory.classLoaderProvider = new ProxyFactory.ClassLoaderProvider() {
            public ClassLoader get(ProxyFactory pf) {
                return Thread.currentThread().getContextClassLoader();
            }
        };
        return ProxyFactory.classLoaderProvider;
    }
}

In our project, we want to support 3rd party modules which should be able to communicate with other modules. For example, a module implementing a business process for crop harvesting needs to be able to retrieve weather information from a weather module implementation. Not only do we need these modules to interact with each other, a lot of data needs to be stored in a database as well.

We basically had two options, the first option was to split up the data model and define the entities in the API module, for example, the "WeatherReport" entity in the Weather API module and the "SensorRun" entity in the Sensor API module. At first, this seemed to be the way to go. We love modularity. But, while we where designing this aproach, we ran into some problems.

Sensor data needs to be linked to a plot. The "Plot" entity was in the area-plot module, as well as in another persistence unit. We foresaw a highly fragmented data model with all kinds of integrity and performance issues. In fact, we think those problems will be bigger than the "big knot in the middle" problem of the second approach, which was a common domain model. Since a lot of modules will be implementing business processes, they will need other modules to provided data.

Whichever option we chose, upgrading to a new model will be difficult. We implemented the second approach by creating a model module containing the core domain entities and their relations. Simplified, we have a "Plot", which can have zero or more "SensorRuns", which can have zero or more "SensorReadings". The sensor-cropcircle module contains a cropcircle-specific sensor implementation. It does not know anything about plots or runs. It does want to store some cropcircle specific attributes which are not in the more generic "SensorReading" entity. So we create a "CropcircleSensorReading" entity and connect it to the "SensorReading" entity from the model module.

The "persistence.xml" in the sensor-cropcircle module only contains two entities, "SensorReading" and "CropcircleSensorReading". The area-plot module imports plots from a shape file, storing them in the "Plot" table,  the "persistence.xml" only contains the "Plot" entity. The link-sensor-plot module links sensor data to plots by creating sensor runs,  the "persistence.xml" contains "Plot", "SensorRuns", and "SensorReadings".

The service layer has a similar hierarchy, an abstract base service in the model module with some functionality to query, store common domain model entities, with specific services in persiting modules wich extend the base service. The API only exposes domain entities and not implementation specific entities like "CropcircleSensorReading".

Decision to Use the NetBeans Platform

A few years ago we were developing an application for a major European harbor. One of the requirements was that the application be web-based. We started to build some parts of the required components in a proof of concept. Soon we discovered that the amount of JavaScript required to implement the amount of user interaction the customer wanted was blowing up the browser. With loads of modules still to come, there was no way we could build a smooth interacting web-based application.

We successfully challenged the web-based requirement. We narrowed down the choices for a rich-client platform to NetBeans and Eclipse. Our team consisted at least half of junior members. The low learning curve of the NetBeans IDE, combined with the usage of standard Swing in the NetBeans Platform made us choose NetBeans. After a short ramp up period, we reached a pretty high development velocity.

Being an architect as well as a developer, the component-based strategy of the NetBeans Platform fitted like a glove. Architectural problems could be adressed and solved very quickly, not leading to any development delay.
Unfortunately, we never got to finish that application, for other reasons.

After being on another project for almost three years, a new development opportunity arose for a Java rich-client application. With the very good experience from the first NetBeans Platform application, we decided to use it again. We were pleasantly surprised with the leaps of progress the NetBeans team had made on the Maven integration, which we decided to use as our build tool.

Advantages of the NetBeans Platform

I really like structured/modular applications. Especially when things get big and complex, structure is required
to keep a decent level of quality. The NetBeans Platform is not only structured and modular itself, but also more or less forces you to structure your own application in that way as well.

As a technical architect, I am responsible for the quality and the structure of the code. The NetBeans Platform structure puts the developers on the team in the right direction, making my job a lot easier. If, for example, a developer asks how he should document his API, I point him to one of the NetBeans API's as reference. Many advantages spin off from the modular design of the NetBeans Platform. The lookup, loose coupling, public packages, module classloaders, etc.

An active community helps with solving problems.

Another advantage is the NetBeans Platform support for Maven. I really started to like and appreciate Maven as a build tool in my previous project. There are still some things to improve, but the foundation is sound.

Technical Tips

Make a design of your modules and stick to it. That will save you a lot of trouble. Everytime I took an architectural shortcut because of time pressure or whatever reason, it backfired.

We created an event bus in our first NetBeans Platform application. We used it in most applications since and it has proven its worth. The Lookup is not always suitable for fire & forget events. Sometimes you just want to shout something into oblivion, not caring if anyone does something with it. Our event bus provides this for us. I could/should probably write a dedicated article about it because it is a bit beyond the scope of this interview.

If you want to peek at our event bus, go here, and consider the article pending. :)

Future Plans

Currently we only have a few "apps" in our store. With the support of public and commercial parties, we want to evolve it into a full flex "farmers IDE". Weather reports, tax forms, fertilizer shops, automated driverless tractors... the sky is the limit.

If anyone is interested in the AgroSense project or has an idea for a related "app", visit the homepage.

You could even make some money with it in the future if farmers like it. And finally, don't worry, be happy, and have great fun building NetBeans rich-client applications!