Load Testing Shiny Apps
Data is ubiquitous. Shiny from R Studio helps to build interactive web applications from R. But performance testing is a little tricky. This article can help!
Join the DZone community and get the full member experience.Join For Free
Data is ubiquitous. Shiny from R Studio helps to build interactive web applications from R. But, performance testing is a little tricky. If you sniff the traffic, you don't get any network data. If you are working on Shiny apps performance testing, then this blog article is for you. This blog article explains load testing shiny apps using
What Is Shiny?
You can develop the shiny app without any advanced development skills. Once the development is done, you can leverage shinyapps.io to deploy the app in the cloud or use the open-source or commercial versions of Shiny Server or RStudio Connect respectively.
Sample Demo App
Here is one of the sample demo apps from the shiny package:
Network Sniffing Shiny App
If we leverage dev tools, it doesn't capture any network calls. Here is the screencast of the sample demo app network capture. We cannot use traditional tools like LoadRunner, JMeter or NeoLoad, etc. Enter
What Is Shiny Load Test?
shinyloadtest is a package which works along with
shinycannon (command-line tool), which enables load testing Shiny apps.
Load Testing Shiny Apps
To load test shiny apps, the following subsections list the pre-requisites.
RStudio is an IDE for R. It comes with a free and commercial version. You can download the open-source edition from here.
Shiny Load Test
You can install the Shiny Load Test package from here. The Shiny Load Test helps to record and analyze the results.
Launch RStudio, create a new project. In the console, enter
install.packages("shinyloadtest") and hit enter.
This will install
shinyloadtest package as shown below:
To run the sample demo app, enter the following commands:
This will launch the demo app locally with a random port as shown below.
Do not close this console. We are going to make use of this to record and load testing using
Shiny Cannon helps to inject the load by replaying the recorded log. To install it in Windows, download the jar file from here.
Let's Start Recording
As you know, the first step is to record the business flow. To begin recording, we are going to leverage the Shiny Load Test package.
Create a new project in RStudio (do not close the previous project where we are running the demo app).
In the Console, enter
library("shinyloadtest") to load the library.
record_session("http://127.0.0.1:5049/"), make sure that you are entering the correct port number where the demo app is running.
This will launch a new tab in your default browser with a different port number.
In the browser, just record your business actions and, once you are done, close the browser to stop the recording.
On the console, you can see the message as shown below.
To view the recording log, open the folder of your RStudio project so you can see the recording.log file. Here is how it looks:
Let's Smoke Test
After the recording, it is always the best practice to run a smoke test to debug the script. To replay the log, we need
Open the command prompt or PowerShell (or Terminal) and navigate to the folder where you have downloaded the Shiny Cannon jar file.
recording.log file to this location as well.
To start the smoke test, enter the below command. This will run the smoke test with 1 thread and store the results in the SmokeTesting folder.
java -jar ..\shinycannon-1.1.0-45731f0.jar .\recording.log http://127.0.0.1:5049 --workers 1 --loaded-duration-minutes 1 --output-dir SmokeTesting
Here is the output of smoke testing:
2021-02-28 22:51:59.648 INFO [thread00] - Detected target application type: R/Shiny 2021-02-28 22:51:59.673 INFO [thread00] - Waiting for warmup to complete 2021-02-28 22:51:59.879 INFO [thread01] - Warming up 2021-02-28 22:51:59.881 INFO [progress] - Running: 0, Failed: 0, Done: 0 2021-02-28 22:51:59.932 INFO [thread00] - Maintaining for 1 minutes (60000 ms) 2021-02-28 22:52:04.941 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:09.943 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:14.943 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:19.955 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:24.956 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:29.960 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:34.962 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:39.963 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:44.966 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:49.966 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:54.967 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:52:59.940 INFO [thread00] - Stopped maintaining, waiting for workers to stop 2021-02-28 22:52:59.967 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:53:04.990 INFO [progress] - Running: 1, Failed: 0, Done: 0 2021-02-28 22:53:07.549 INFO [thread01] - Stopped 2021-02-28 22:53:07.553 INFO [thread00] - Complete. Failed: 0, Done: 1
We are good as we are not seeing any failures in our smoke testing. Now, let us inject more load into the demo app.
Let's Start Load Testing
As we are running the app locally, it is not a good idea to inject a high load. Just to demonstrate, I am going with 2 workers for 2 minutes. Issue the below command and monitor the output for any issues.
After the testing is completed, the next step is to generate the test results.
Go back to the RStudio project to issue a few commands to generate the HTML report. In the below command, store the run to
df <- load_runs('Run1' = 'C:/Tools/Shinyapp/ShinyAppLoadTest/LoadTestRun1')
To plot the results in RStudio, you can enter the below commands:
slt_http_latency(df) slt_hist_loadtimes(df, max_load_time = 5) slt_session(df) slt_websocket_latency(df, cutoff = 5)
To generate the HTML report, enter the below command:
This will generate all kinds of graphs, plots, tables, etc. You can find the HTML report in the project folder. Here is the waterfall chart:
Here is the event duration chart:
Congratulations! Now, you have successfully completed a load test of Shiny app.
The Shiny Load Test comes with its own limitations, in terms of deployment and application types.
You cannot record or load test the shiny app which is deployed at shinyapps.io.
You need WebSockets and recordings are server-dependent.
R Markdown documents with
runtime: shiny are unsupported.
Apps with high latency may cause failures.
I suggest you look out for the issues in the GitHub repo.
If you are working on Shiny apps, you definitely need to check out Shiny Load Test. This package is kind of fresh out of the oven, so it may not fulfill all of your requirements. But, this is a good start for load testing shiny apps. Also, you can check out my other article which uses JMeter under the hood.
Published at DZone with permission of NaveenKumar Namachivayam, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.