Keeping Your Balance: Load Balancing Quartz Cron Jobs With Red Hat JBoss Fuse

In the beginning of the year I posted a post on how to do cron jobs with Apache Camel in JBoss Fuse, the reason I am doing it is because sometimes we need to off-load the integration jobs and schedule them to run in less time during busy hours. Cron Jobs allows you to run the integration application at the certain period of time you set it to run, it will be mostly in the night time. We use the Quartz2 component in camel to achieve this.

Just to recap a little on the scenario I had. Basically it starts up two instance that runs the same cron jobs, the job simply writes a file to an FTP server. The job is implemented by Quartz2 and setting it to run twice every minute,both instances are connected to a database (In my code I used H2 for easy installation, but there is another option to use PostgreSQL database, see my previous blog). The reason we want to connect them to database is to make sure these two instance will share the loading of the the job. This is very useful if you have a long running job, and can divide them into different parts and share the loading among distributed resources.

Don't forget to setup the database beforehand, setting the database table schema, download the Quartz library, you will find the scripts to setup tables are under docs/dbTables, find the sql file of the database you are using. Run the sql file to create all the tables needed in your database.

And let's go back to the camel project itself. The setting of the database are in the properties.

<bean id="quartz2" class="org.apache.camel.component.quartz2.QuartzComponent">
    <property name="propertiesFile" value="org/blogdemo/demojob/myquartz.properties"/>

</bean>

Since we are going to deploy this with fabric, we will place the properties file under main/fabric

Inside myquartz.properties contains detail about the shared database such as driver, ID , password and also the setup detail like, number of threads, driverDelegateClass used and most importantly if this setting is clustered.

# Main Quartz configuration
org.quartz.scheduler.skipUpdateCheck = true
org.quartz.scheduler.instanceName = DatabaseClusteredScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.jobFactory.class = org.quartz.simpl.SimpleJobFactory
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.dataSource = quartzDataSource
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.jobStore.clusterCheckinInterval = 20000


# JobStore: JDBC jobStoreTX
org.quartz.dataSource.quartzDataSource.driver = org.h2.Driver
org.quartz.dataSource.quartzDataSource.URL = jdbc:h2:file:~/h2/cronjob;AUTO_SERVER=TRUE
org.quartz.dataSource.quartzDataSource.user = sa
org.quartz.dataSource.quartzDataSource.password = 

org.quartz.dataSource.quartzDataSource.maxConnections = 5

The camel route is very simple and easy, 

<route id="quartzRoute">
        <from uri="quartz2://myGroup/myTimer?cron=0/45+*+*+*+*+?"/>
        <setHeader headerName="CamelFileName">
            <simple>MyJob-${date:now:yyyyMMdd HH:mm:ss}.txt</simple>
        </setHeader>
        <setBody>
            <simple>${date:now:yyyyMMdd HH:mm:ss} Hello This is the content from Quartz2 - {{instancename}}</simple>
        </setBody>
        <to uri="sftp://demo@localhost?password=ZAQ!2wsx"/>
    </route>

Basically, it is kicked of by the cron job, on every 0 and 45 second, and then creates a file under name MyJob-YYYYMMDDHHMMSS.txt with content Hello This is the content from Quartz2 in the file. 

And to deploy the application to JBoss Fuse, I need to create a fragment bundle for connection pooling, here we use c3p0 and the JDBC driver. And then deploy both onto the same instance. 




Deploy another container with the same job profile and fragment bundle. 



Add another profile,  that prints the content received in FTP server. 





If you encounter any problem when deploying, it's probably OSGi not loading the bundles correctly, you can simply restart the container or make sure the following bundles are activated in the container. (Check by using osgi:list) 
[ 157] [Active     ] [            ] [       ] [   80] Apache ServiceMix :: Bundles :: c3p0 (0.9.1.2_1), Fragments: 148
[ 167] [Active     ] [            ] [       ] [   80] H2 Database Engine (1.4.181)
[ 172] [Active     ] [Created     ] [       ] [   80] Quartz Two Demo (1.0.0.SNAPSHOT)


Take a look at the log running quartzprint profile, you will find the jobs are clustered and loads are shared between two nodes. 




And detail of the job can also be found in console of each container.



You can find the Demo code and detail instructions here.