Analyse Mule4 API Performance — Using JMeter and Google Apps Script

For a project member, when we heard about API performance tests. Do we always have some questions like what should our standard for API performance? How we can define an automation process for API performance tests? What are the tools we can use? 

We also had these questions for a project and I come with a solution, which works for us and helped us in terms of time and test effort. So, I would like to share that experience using this blog.

I was working on a project where we have a huge number of APIs and those have to pass the defined standard of performance tests. But, we didn’t have an automation process for them. So, I came with an idea to make this performance test process automatic and prepared a Performance Test Framework using Jmeter and Google Suit.

Requirements

• Perform the load/performance test for each E2E Mule API chain and prepare the output.
• Prepare the output properly so that it will be easy for the next developer to easy to analyze the test outputs.
• Based on the aggregated reports of multiple tests (with an incremental load), we have to plan and take the next action to improve the performance of those APIs.

For the Above Requirement, I Choose Below Technologies 

1. Mulesoft
2. Jmeter
3. Google App Script

Sample Mule 4 API

This API is just a sample to show this use case. This sample API is used to create employee details by storing them in a local MySQL database. Here is the RAML definition.

I develop this API using Mule 4 and can be available here.

JMeter Configuration for API Test

We can install Jmeter from here. As soon as we installed the Jmeter, we can import the JMeter script from here. As I develop a simple API without any authentication, so I don't have to configure any HTTPS or other authentication properties in the Jmeter script, but if your requirement needs any configuration you can follow here also.

Here is the Jmeter test script for the Above Mule API.

How to Perform Test Using JMeter

1. As soon as we configured the JMeter script and deployed the Mule Application, we have to decide on the test configuration. For this we have to finalize the below 4 parameters:
   • How many cycles of tests we are going to perform.
   • How many Number of Threads (users) we are going to use(increase) for each test.
   • What would be the Ramp-up period (seconds) for each thread creation?
   • What would be the total duration of each test?
2. Once we decide these, we have to identify the output format of the JMeter test. As per the attached JMeter script, the output will be stored in results.csv (sample) format. And this CSV file will contain the Columns as below:- 
3. Kindly don't try to modify the output format and the column position, as I wrote the Google Script based on these column's position.
4. Then we have to decide the location to store the test output(CSV) file.
5. For Error Logging I have to define a test-script.jmx (sample) file, which will record the details of the error output. We have to decide the location for that. 
6. Once we ready with this configuration, we can start executing the test. For example, I have performed 5 cycles of tests, by increasing the Thread like 100, 150, 200, 250, 300. I kept the ramp-up period as 5 sec for all 5 cycles. I have executed each test for 120 sec. Based on your requirement you can change this configuration.
7. Once the 1st test is done and the output is stored in results.csv, cut and save that file into another folder by giving a name like "100_user-5_sec_rampup.csv". Once the 1st test cycle is done, start executing the 2nd test cycle and keep that result csv like "150_user-5_sec_rampup.csv". Keep repeating the tests and store all the 5 cycle output CSV files into a separate folder. 
8. Once the test outputs(files) are ready, we can use google script to analyze the output, and based on our analysis, we can take our next action. Like fine-tune our API, increase RAM, increase vCore, or do nothing.

Google App Script to Analyse the Output of Our Test

If our project allows us to use Google Drive and Google App Script then only this Automation Framework is useful. My end goal is to make an Automation performance Framework, using which any project can easily analyze their API's performance and do their next action plan according to their needs.

For this, I have written an App script that will use the Jmeter test output (CSV) files and create a Chart for better analysis.

Steps to Use Google App Script

1. We have to use google drive to use this framework. Create a folder in your google drive with the name TestsOutputs and another folder name RawResults (inside the "TestsOutputs" folder).
2. Keep the JMeter tests output CSV files in the RawResults folder.
3. Copy the google sheet from here and stored in the TestsOutputs folder, then click on tools -> Script Editor.
4. You will land on the Google Apps script tool, then add the script from here. 
5. Once you copied the script, click on run in the google script tool. It will ask your permission to copy the stored CSV files from the RawResults folder to your google sheet and execute the Chart creation process. Once you grant permission, you can view the execution logs. Here is the sample of google script.
6. As soon as the execution finishes it will create two separate Charts. One for each test output and one is for aggregated test output. 
7. Sample of individual test output 
8. Sample of aggregated tests output
9. Based on these output charts we can have a clear picture of our API performance and can decide the next action based on project requirement.

Author's Comment

• This Performance Test Framework is more useful for those projects who has a requirement of huge number of of user hits and who must need performance test for their APIs before it went for live.
• This framework is useful for those project who allows using google suits. 
• This framework can easily shareable across the organization or within allowed project groups with full security (as google provides permission facilities at each level).
• I would be happy to receive your valuable feedback about this topic.