DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • Java, Spring Boot, and MongoDB: Performance Analysis and Improvements
  • Spring Data: Easy MongoDB Migration Using Mongock
  • MongoDB With Spring Boot: A Simple CRUD
  • Multi-Tenancy Implementation Using Spring Boot, MongoDB, and Redis

Trending

  • Operational Principles, Architecture, Benefits, and Limitations of Artificial Intelligence Large Language Models
  • How Can Developers Drive Innovation by Combining IoT and AI?
  • Analyzing Techniques to Provision Access via IDAM Models During Emergency and Disaster Response
  • Can You Run a MariaDB Cluster on a $150 Kubernetes Lab? I Gave It a Shot
  1. DZone
  2. Coding
  3. Frameworks
  4. Clustered Quartz Scheduler With Spring Boot and MongoDB

Clustered Quartz Scheduler With Spring Boot and MongoDB

In this writing, I will be discussing how to achieve single execution in a clustered environment using Quartz. Do not be frightened of Quartz though.

By 
Amanuel G. Shiferaw user avatar
Amanuel G. Shiferaw
DZone Core CORE ·
Jan. 23, 21 · Tutorial
Likes (3)
Comment
Save
Tweet
Share
26.6K Views

Join the DZone community and get the full member experience.

Join For Free

Spring Boot library for Quartz does not work correctly if you start up two instances of service in parallel. Each one of the services starts the execution of the same jobs while the expected behavior is one of the instances is elected to execute the jobs and the other ones waiting in the background, and in the case of failure of the first service, another one is elected.

Springs’ @Scheduled annotation is a very convenient and easy way of scheduling tasks for several reasons. But, if you have a clustered environment and you must run any job just from one node, that does not hold true. You realize that all scheduled jobs will start executing almost at the same time. This may cause problems ranging from unnecessary calls to data inconsistency.  

Searching for a solution to achieve this behavior, I found ShedLock and Quartz as prominent once while the latter one is much more robust and can be used in any Java application and so most widely used, while the former one is much easier to configure.  

You can get a good introduction to ShedLock from here.

In this writing, I will be discussing how to achieve single execution in a clustered environment using Quartz. Do not be frightened of Quartz though.

So, let us start by adding dependencies to the application. By default, Quartz only provides support for the traditional relational databases. But, thanks to Michael Klishin and MuleSoft, who created a MongoDB implementation of the Quartz library in a clustered environment (can be found here over GitHub).

XML
 




xxxxxxxxxx
1
11
9


 
1
<dependency>
2
    <groupId>org.springframework.boot</groupId>
3
    <artifactId>spring-boot-starter-quartz</artifactId>
4
</dependency>
5
<dependency>
6
  <groupId>com.novemberain</groupId>
7
  <artifactId>quartz-mongodb</artifactId>
8
  <version>2.1.0</version>
9
</dependency>



I have set up a config like the following in my application.yml. You can choose a different structure or use the default spring.quartz.YYYY. It is a bit verbose indenting all those properties (laziness is evil though), so I choose this one. You can check all the scheduler properties over here.

YAML
 




xxxxxxxxxx
1
17


 
1
quartz:
2
  properties:
3
    org.quartz.scheduler.instanceName: MyClusteredScheduler
4
    org.quartz.scheduler.instanceId: AUTO # you also can define a custom generator of your own or use already existing once like HostnameInstanceIdGenerator
5
    org.quartz.scheduler.skipUpdateCheck: true
6
    org.quartz.scheduler.jobFactory.class: org.quartz.simpl.SimpleJobFactory
7
    org.quartz.scheduler.threadsInheritContextClassLoaderOfInitializer: true
8
    org.quartz.threadPool.threadCount: 2
9
    org.quartz.threadPool.threadPriority: 5
10
    org.quartz.jobStore.isClustered: true #here is the magic which allows creating a lock automatically. Check here for more 
11
    org.quartz.jobStore.misfireThreshold: 30000
12
    org.quartz.jobStore.class: com.novemberain.quartz.mongodb.MongoDBJobStore
13
    org.quartz.jobStore.mongoUri: ${spring.data.mongodb.uri}
14
    org.quartz.jobStore.dbName: ${your_db_name}
15
    org.quartz.jobStore.collectionPrefix: qrtz
16
    org.quartz.jobStore.clusterCheckinInterval: 60000 # checks-in with the other instances of the cluster, default 15000



As you have seen I made theorg.quartz.jobStore.isClusteredto be true. As per the documentation clustering currently only works with theJDBC-Jobstore (JobStoreTX or JobStoreCMT), and essentially works by having each node of the cluster share the same database.

Load-balancing occurs automatically, with each node of the cluster firing jobs as quickly as it can. When a trigger’s firing time occurs, the first node that acquires a lock will fire it.

Moving on, we then need to let Spring manage the creation of the Quartz scheduler using the SchedulerFactoryBean.  In addition, load all the properties from application.yml

Java
 




xxxxxxxxxx
1
34


 
1
@Configuration
2
@ConfigurationProperties(prefix = "quartz")
3
public class QuartzConfiguration {
4

          
5
    private Map<String, String> properties;
6

          
7
    public Map<String, String> getProperties() {
8
        return properties;
9
    }
10

          
11
    public void setProperties(Map<String, String> properties) {
12
        this.properties = properties;
13
    }
14

          
15
    private Properties getAllProperties() {
16
        Properties props = new Properties();
17
        props.putAll(properties);
18
        return props;
19
    }
20
    
21
    // Don't Inject this directly in other spring Components since it is a factory, instead use Scheduler
22
    @Bean
23
    public SchedulerFactoryBean schedulerFactoryBean() {
24
        SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
25
        scheduler.setQuartzProperties(getAllProperties ());
26
        scheduler.setWaitForJobsToCompleteOnShutdown(true);
27
      // Set the key of an ApplicationContext reference to 
28
      // expose in the SchedulerContext. If you extend QuartzJobBean instead, 
29
      // you have to use setApplicationContext
30
     scheduler.setApplicationContextSchedulerContextKey("applicationContext"); 
31
        
32
        return scheduler;
33
}



The job to be executed follows. Note that it is recommended not to add any business logic unless necessary and related to the job itself.

Java
 




xxxxxxxxxx
1
25


 
1
public class SyncJob implements Job {
2

          
3
    @Override
4
    public void execute(JobExecutionContext context) throws JobExecutionException {
5
        try {
6
            ApplicationContext ctx = getContext(context);
7
            SyncService SyncService = 
8
              (SyncService) ctx.getBean(SyncService.class);
9
            SyncService.start();
10
        } catch (Exception e) {
11
          // do something
12
        }
13
    }
14
    
15
    private ApplicationContext getContext(JobExecutionContext context) throws Exception {
16
        ApplicationContext ctx = 
17
          (ApplicationContext) context.getScheduler().getContext().get("applicationContext");
18
        if(ctx == null)  {
19
            throw new JobExecutionException("No application context available in scheduler context.");
20
        }
21
        return ctx;
22
    }
23
}



And finally, let us define Jobs and their Triggers.

Java
 




xxxxxxxxxx
1
35


 
1
@Configuration
2
public class JobConfiguration {
3
    private static final String SYNCDATAJOB = "syncDataJob";
4
    private static final String SYNCDATAGROUP = "syncData_group";
5
    private static final String SYNCDATATRIGGER = "syncData_Trigger";
6

          
7
    @Autowired private Scheduler scheduler;
8
    
9
    @PostConstruct
10
    private void init() throws Exception {
11
        scheduler.addJob(syncData(), true, true);
12
        if (!scheduler.checkExists(new TriggerKey(SYNCDATATRIGGER, SYNCDATAGROUP))) {
13
            scheduler.scheduleJob(triggerSyncData());
14
        }
15
    }
16
    
17
    private JobDetail syncData() {
18
        JobDetailImpl jobDetail = new JobDetailImpl();
19
        jobDetail.setKey(new JobKey(SYNCDATAJOB, SYNCDATAGROUP));
20
        jobDetail.setJobClass(SyncJob.class);
21
        
22
        return jobDetail;
23
    }
24
    
25
    
26
    private Trigger triggerSyncData() {
27
        return newTrigger()
28
                .forJob(syncData())
29
                .withIdentity(SYNCDATATRIGGER, SYNCDATAGROUP)
30
                .withSchedule(simpleSchedule()
31
                        .withIntervalInMilliseconds(60000L))//hours, min, sec, millis
32
                .build();
33
    }
34
}



Finally, the service to be called upon job execution:

Java
 




xxxxxxxxxx
1
11


 
1
@Component
2
public class SyncService {
3

          
4
    private Logger logger = LoggerFactory.getLogger(SyncService.class);
5

          
6
    public void start() {
7
        logger.info("{} Requesting for a new token", "SyncService#start()");
8
        //do the logic
9
    }
10
}



When you now run the Spring Boot application you should see the following collections created:

  • quartz_calendars
  • quartz_jobs
  • quartz_locks
  • quartz_schedulers
  • quartz_triggers

Note: If you are interested to have a custom job store you can follow this article.

If you read this far, it implies that you have enjoyed the article. Thank you.

Spring Framework Spring Boot Quartz (scheduler) job scheduling clustered career MongoDB

Opinions expressed by DZone contributors are their own.

Related

  • Java, Spring Boot, and MongoDB: Performance Analysis and Improvements
  • Spring Data: Easy MongoDB Migration Using Mongock
  • MongoDB With Spring Boot: A Simple CRUD
  • Multi-Tenancy Implementation Using Spring Boot, MongoDB, and Redis

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!