Automate and Persist MuleSoft CloudHub Application Logs
Learn how to write a shell script that will take two hours of work with application logs and turn it into a two minute process.
Join the DZone community and get the full member experience.Join For Free
Hey guys. This article will help us to understand working with Anypoint CLI 3.x commands, which interacts with MuleSoft's Anypoint Platform.
As CloudHub has a specific log retention policy, it stores up to 100MB of logs per app and per worker or for up to 30 days, whichever limit it hits first. Once this limit has been reached, the oldest log information is deleted in chunks and is irretrievably lost.
To overcome this problem, MuleSoft application logs from CloudHub are manually downloaded every day and stored in a storage location (Box, Amazon S3, etc.). This activity is taken care of by developers and operation team members.
The most common negative impact of this manual log downloading process is the amount of effort and time it takes. It usually takes around two hours of time to download 30+ MuleSoft application logs. Developers don't like spending too much time on this type of work and, as a result, it can have a costly impact on your entire team.
The script we'll write in this article will automate two plus hours of work into a process that takes just two minutes.
What Is Anypoint CLI?
According to the MuleSoft docs, "Anypoint Platform provides a scripting and command line tool for both Anypoint Platform and Anypoint Platform Private Cloud Edition (Anypoint Platform PCE). The command line interface (CLI) supports both the interactive shell and standard CLI modes."
What Is CloudHub?
Per MuleSoft, "CloudHub is the platform as a service (PaaS) component of Anypoint Platform."
What Is Box and Box CLI?
Box is a cloud computing business which provides file sharing, collaborating, and other tools for working with files that are uploaded to its servers. Users can determine how their content can be shared with other users. Users may invite others to view and/or edit an account's shared files, upload documents and photos to a shared files folder (and thus share those documents outside Box), and give other users rights to view shared files. The Box Command Line Interface (CLI) is a tool for making requests to Box APIs from your terminal window or command prompt.
What Are We Trying to Achieve?
We are going to create a shell script which will interact with Anypoint Platform's Runtime Manager, using Anypoint CLI commands and, later, connect to Box, using Box's CLI commands.
This script will Download MuleSoft application logs from CloudHub, for all provided applications, and later upload it to Box.
- Anypoint Platform Developer Account --> https://anypoint.mulesoft.com
- Box Developer Account --> https://account.box.com
- Anypoint CLI Setup --> https://docs.mulesoft.com/runtime-manager/anypoint-platform-cli
- Box CLI Setup --> https://developer.box.com/guides/tooling/sdks/cli/
- First of all, you will need to login to your Anypoint Platform account. Make sure you have some applications deployed in your respective Business group.
- Install Anypoint CLI on your machine. You will get all the installation steps here.
- Now sign up for Box here. Once this is done, follow the below steps:
3. Once this is created, it has to be authorized by Admin. After it's been authorized, you will be able to see this app in the Admin portal.
Check the detailed of steps for box configuration here.
1. Anypoint CLI
Below are some basic Anypoint CLI commands; you can checkout the full list of commands here.
1. runtime-mgr cloudhub-application list [options] This command lists all applications available in your Anypoint Platform CLI. 2. runtime-mgr cloudhub-application start [options] <name> This command starts the running application you specify in
3. runtime-mgr cloudhub-application download-logs [options] <name> <directory>
This command downloads logs the for application specified in
<name>to the specified directory
2. Box CLI
Below are some basic Box CLI commands; you can checkout the full list of commands here.
1. box folders:get 111111 This command fetch the information about a folder with an id 111111. 2. box files:upload fileName --parent-id 100 This command upload's a file to a specified folder ID. 3. box files:download 11111 --destination /path/to/destinationFolder This command download a file from box folder & save it to destination folder.
Let's first check the deployed application in CloudHub. I have deployed five sample applications.
Now, let's go through the script and understand it. I have added commands inside the script to make it self-explanatory.
- I have added lot of indentation and commands inside the script to dress up the output in the console, because of which the script might look a little bigger.
- Below is an initial version of the script. We can enhance this script by considering parameters like exception handling, logging, best practices, etc.
Once the above script is executed on any bash shell, you will get a similar kind of response in the console.
At last, you will see application logs are getting uploaded successfully to Box.
This is just one of the use cases for Anypoint CLI. This script can be automated using cron to let it run at a specific date/time.
Anypoint CLI is a very powerful interface that can be used to automate and accomplish tasks using the command line tool.
Opinions expressed by DZone contributors are their own.