Scheduling Jobs on Heroku with Azure Logic Apps
Learn to use Azure Logic Apps to run scheduled tasks on Heroku at minimum cost
Join the DZone community and get the full member experience.Join For Free
Many times, your application needs to run tasks or jobs on regular intervals. Scheduling might be necessary when polling an API every night, or dispatching emails with reports each week. And sometimes you may find that you need to trigger a host of tasks across multi- or hybrid-cloud and need a way to run these tasks in a reliable, cross-cloud manner.
In this article, let’s look at one way to do that using a Heroku dyno for our task, and Microsoft Azure Logic Apps as our scheduler service. The scheduler in Logic Apps allows for triggering workflows that include services across clouds and on-premises, paving the way for easy administration and observability of many disparate tasks. Functionally, the scheduler not only lets you specify simple recurrence rules (such as every minute), but also complex ones (such as every 15 minutes during work hours).
Let’s discuss the Azure Logic Apps service in detail next.
Job Scheduling with Azure Logic Apps
The Azure Logic Apps cloud service helps you automate and orchestrate workflows on the cloud. With Azure Logic Apps, you can build enterprise integration solutions using the many connectors available for popular cloud services such as SAP, Oracle DB, and Salesforce.
Azure Logic Apps support running recurring tasks and processes on a schedule, using the schedule trigger. You can read the Microsoft guide to understand the Logic Apps support for executing recurring tasks. The following list offers some scheduling patterns that can be implemented using Azure Logic Apps (source: Microsoft).
- Run immediately and repeat every n number of seconds, minutes, hours, days, weeks, or months.
- Start at a specific date and time, then run and repeat every n number of seconds, minutes, hours, days, weeks, or months.
- Run and repeat at one or more times each day, for example, at 8:00 AM and 5:00 PM.
- Run and repeat each week, but only for specific days, such as Saturday and Sunday.
- Run and repeat each week, but only for specific days and times, such as Monday through Friday at 8:00 AM and 5:00 PM.
One great thing about using the Azure Logic Apps is that you only pay for the execution of actions, ensuring money is not wasted on services sitting idle.
Scenario: One-off Dyno Image Deliveries
Unsplash has a rich library of images and REST APIs to fetch them on demand. It would be great to have them mailed to me every day. I am going to create an application named Picletter, which I will host as a one-off dyno. When executed, this application will fetch the day’s image from Unsplash and use the SendGrid email service to send me the picture in an email. To schedule the Picletter application’s execution, I will create an Azure Logic App with a schedule trigger configured to execute every day. Upon execution, the Logic App will use the Heroku Platform API to run the Picletter application.
The following sequence diagram illustrates the workflow:
For reference, the source code of the application and the Azure Logic App deployment template is available in my GitHub repository here.
Let’s now build and deploy the Picletter app to Heroku.
Heroku documentation outlines several types of dynos that you can use to host relevant workloads. Heroku recommends using one-off dynos for executing scheduled jobs in a disconnected manner. That’s because with one-off dynos, you only pay for the time your dyno spends processing data, instead of both active and idle time.
There are three ways to interact with Heroku: the User Interface, Platform API, and the Heroku CLI. Due to its simplicity, the CLI is my preferred tool to create and manage Heroku apps. Let’s begin the process of creating the Picletter Heroku application by executing the following command. Remember to choose another name for your application if the one below is not available.
For inner loop development, I configure my CLI tools to produce verbose output to quickly identify and fix any issues. To enable verbose output from the Heroku CLI, set an environment variable
HEROKU_DEBUG with value 1.
Note that on the execution of the previous command, Heroku generated a remote Git repository for us. On every push of your application code to this repository, Heroku will attempt to build and deploy your application. Create a folder on your system to store the application code, launch the terminal, and change directory (
cd) to the folder. Next, execute the following commands to set up a local Git repository, and set the Heroku-hosted Git repository as the upstream remote repository.
After the integration is set up, we will be able to commit and push code to the remote repository, and Heroku will attempt to deploy the application on every push. Let’s build the Picletter application now. I am going to use Go, which is one of the programming languages supported by Heroku. I will not discuss the basics of building and deploying Golang apps to Heroku, only because Heroku already has an excellent “getting started” guide on writing applications with Go and deploying them to Heroku.
Let’s begin with creating a Go module. Execute the following command to create a new module named picletter.
After execution, the command will generate a module file named go.mod that contains the module’s name, the Go version, and the dependencies used by the module. To fix the version of Go used by Heroku for building the module, add the following comment to the go.mod file, after which the module file should look like the following:
Execute the following command to install the necessary packages for our application. We will install the SendGrid Go packages to prepare and send emails and a library named imgbase64 to download an image from Unsplash and convert it to Base64 format.
Let’s now create a file named main.go and populate it with the following code. We will first use the Unsplash API to fetch the daily image and then use the SendGrid library to prepare an email containing the image. Finally, we will use the SendGrid library to send the email to the receiver. I encourage you to read the official SendGrid Golang SDK documentation to understand the library’s usage and instructions on fetching the SendGrid API Key that you will use to authorize API calls to SendGrid.
You must have noticed that we used a couple of environment variables to configure the application. Heroku supports storing the configuration values (called config vars) specific to your app’s deployment in a separate configuration registry. With individual configuration registries, you can set different values of the same variable for different environments. For example, the connection string for the dev deployment can point to the dev database, and that same variable when used in the prod deployment can point to the prod database. Heroku makes the config vars available as environment variables to your application.
Let’s execute the following command to set the values of the config vars/environment variables for our application.
Heroku doesn’t yet know how it can start our application. We’ll add a Procfile that specifies the command Heroku will use to launch our application.
Just like a gitignore file is used to specify intentionally untracked files in Git, you can use a slugignore file to keep unnecessary files from being deployed on the Heroku dyno. I don’t want to include the Azure Logic App ARM template and the README file in the application build artifacts, which is why I have added the following specifications to the .slugignore file:
We are now ready to deploy the Picletter app to Heroku. Let’s now push the code to the Heroku Git repository, which will kick off the application build and deployment on Heroku.
Let’s navigate to the Heroku dashboard to visually inspect the deployment and the config vars we set earlier.
After receiving a response indicating success from the command, we can navigate to the email inbox that we specified as the value of the
RECEIVER_EMAIL config var. The following screenshot shows the email that I received after executing the Picletter application.
We used the Heroku CLI to trigger the previous dyno run. Let’s now use Azure Logic Apps and Heroku Platform API to automate the execution of the application on a schedule.
Scheduling Picletter Application Run with Azure Logic Apps
Let’s deploy the Logic App with the Azure Resource Manager (ARM) template available in the logic-app folder in the repository. The ARM template defines a workflow triggered by a recurrence trigger (called schedule trigger) scheduled for execution every day. The schedule trigger executes an HTTP action that sends a POST request to the Heroku Dyno Platform API to run the one-off Picletter dyno. Remember to update the value of the ARM template’s
uri parameter so that it mentions the name of the Heroku app you created earlier. Also, note that you can specify the SKU of the dyno that should host the Picletter job in the HTTP request body.
All resources in Azure must exist under a resource group. So, let’s create a resource group named picletter-rg and use the ARM template to create a Logic App within the resource group with the Azure CLI commands below. Note that we must pass the name of the Logic App and Heroku Platform API authorization token as parameters to the template. To generate a token that you can use to authorize your Platform API calls, refer to the authentication section of the Platform API quickstart guide.
After executing the commands, let’s visit the Azure management portal to inspect the Logic App we just created. Since we did not configure the start time of the recurrence schedule, you will notice that the workflow executes soon after we deploy it and every 24 hours afterward. Click on the latest workflow run in the Runs History section to view the details of the run.
We discussed the pros and cons of a few strategies that you can use to execute scheduled jobs on Heroku. By combining the services of Azure and Heroku, we built an easy-to-maintain, simple solution that is cost-effective and reliable. As a further improvement to the sample, you can add support for sliding window triggers to retry operations for any missed execution windows.
If you are requesting Heroku to run the one-off dyno in detached mode from the Logic App, you can also try recording the progress or status of a one-off dyno run using the dyno info API. Using the details that you receive, you can raise alerts or update the Logic App to cancel a run if a dyno is already in an active state from the previous run.
I hope you enjoyed building this sample and learned about simplifying an integration scenario using different cloud services.
Published at DZone with permission of Rahul Rai, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.