Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Tick Tock: An Alternative View to Scheduling Jobs

DZone's Guide to

Tick Tock: An Alternative View to Scheduling Jobs

RunDeck is a strong open-source job scheduler. Initially, we thought it was going to be our choice for running planned jobs. However, we were able to meet our needs another way.

· DevOps Zone
Free Resource

The DevOps Zone is brought to you in partnership with Sonatype Nexus. The Nexus Suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

With an effort underway to break up a legacy, monolith application into a collection of RESTful microservices, our engineering team realized that a shared scheduling service was going to be required as part of the project. In order to fully migrate from the legacy platform, there were jobs that needed to run on a periodic basis, which handled the business needs of our customer.

Looking for a Job Scheduler

Faced with this need, our group did what most probably do in this position: start looking at job scheduling applications and frameworks. In talking with our DevOps team manager, we were told that instances of a couple schedulers were already in place. One of them ran exclusively on the Windows platform, while the other was RunDeck, an open-source solution running on Linux.

After some discussion, we decided to look at the RunDeck application in order to see if the open-sourced solution will meet our needs. Our goal was to keep everything contained within the microservice and have the job scheduler simply call a RESTful API at a given date or time.

We were able to prove this out and things looked like they were on track.

Diving Deeper

As we continued to look deeper into the RunDeck solution, we realized there were some challenges with the RunDeck approach that were coming to the forefront of our evaluation:

  • The microservice needed to perform the necessary API calls to schedule the jobs as part of the build and deploy process (including removal of any already scheduled jobs).

  • There was need for the build/deploy process to validate only one instance of each scheduled job was in place at a given time (no duplicates can exist)

  • We needed to figure out security for the identity calling the API and were not happy about using RESTful API calls more like remote procedure calls.

Our goal was to make RunDeck as unaware as possible of our microservice and the scheduling needs that were required. In our view, this was one less thing to worry about.

While we found ways to accomplish each bullet point above, things started to get less tidy...at least in our view. This was not the fault of RunDeck, as it seems like a pretty nice scheduler. Instead, this was a vision our team had in mind for how we want to integrate with a scheduler.

Matt's Idea: Introducing Tick Tock

Matt, one of my team members, started running with an idea of using RabbitMQ and a small Java application that would publish events as topics on a queue. He called the application "Tick Tock" and built it using the Spring Boot/Netflix OSS ecosystem we already have in place. At a high level, there is a service that uses a Cron-based configuration to place messages on a message queue for time intervals we want to publish using topics we have created.

As an example, we want to place messages on the queues every minute, five minutes, ten minutes, half-hour, hour, six hours, twelve hours, and each day. Then, the microservice applications listen to the queue in order to know if there is a trigger to indicate work needs to be executed.

Matt's idea basically published messages on Rabbit MQ following a schedule, that was built with Cron behind the scenes. The message that appears every five minutes (0 */5 * * * *) would be called cron.minute.5 and posted for applications to listen for that topic using Spring AMQP.

Using Java, a simple example implementation is shown below:

@RabbitListener(bindings = 
  @QueueBinding(value = @Queue(value = "${spring.application.name}.worker.<fiveMinuteJob>",
                               durable = "false",
                               arguments = {@Argument(name = "x-message-ttl",
                               value = "30000",
                               type = "java.lang.Integer")}),
                exchange = @Exchange(value = "our.scheduler",
                                     ignoreDeclarationExceptions = "true",
                                     type = ExchangeTypes.TOPIC,
                                     durable = "true"),
                key = "cron.minute.5"))
public void fiveMinuteJob(@Header("timestamp") Date timestamp) {
     // Start coding your job here
}

With the our.scheduler" RabbitMQ queue up and running, the fiveMinuteJob() method would be listening to the cron.minute.5 topic for a new message to be placed on the queue. When a message is found, the fiveMinuteJob() code would fire.

The Results

By implementing Matt's approach, the microservice does not need to worry about maintaining jobs or the identity of the person running the job on a secondary system, like RunDeck. Additionally, no RESTful endpoints need to be exposed for the scheduler to call, keeping our RESTful APIs closer to the REST specification.

The only challenge is to make sure that there is a topic in place that meets the needs of the application. Perhaps, there is a need for a job to run every eight hours (not listed above), which would simply require a new item in the Tick Tock application configuration.

Finally, all the business logic remains in the microservice. Of course, there may be a new challenge with knowing what to do every five minutes, but that program code will always exist within the microservice, instead of being a special job configuration in a job scheduler service like RunDeck.

Conclusion

RunDeck is a very strong open-source job scheduler. Initially, we thought it was going to be our choice for running planned jobs. However, using Matt's lead, we were able to creatively meet our needs in another manner, which appears to be the better choice in our environment.

I am interested to hear your thoughts on our approach. Have you tried something along these lines? If so, what results did you encounter?

The DevOps Zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today

Topics:
devops ,rundeck ,job scheduling ,spring boot

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}