DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Why Use AWS Lambda Layers? Advantages and Considerations
  • Monitoring and Logging in Cloud Architecture With Python
  • The Essentials of Amazon S3 Glacier for Affordable and Compliant Long-Term Data Archiving
  • Is It Okay To Stop Running Your Tests After the First Failure?

Trending

  • The Human Side of Logs: What Unstructured Data Is Trying to Tell You
  • How the Go Runtime Preempts Goroutines for Efficient Concurrency
  • Transforming AI-Driven Data Analytics with DeepSeek: A New Era of Intelligent Insights
  • AI, ML, and Data Science: Shaping the Future of Automation
  1. DZone
  2. Coding
  3. Languages
  4. Upload Files to AWS S3 in JMeter Using Groovy

Upload Files to AWS S3 in JMeter Using Groovy

There are three primary reasons for choosing AWS S3: affordability, speed, and reliability. Here, learn how to upload files to AWS S3 in JMeter using Groovy.

By 
NaveenKumar Namachivayam user avatar
NaveenKumar Namachivayam
DZone Core CORE ·
May. 05, 22 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
5.7K Views

Join the DZone community and get the full member experience.

Join For Free

I use my personal AWS S3 to store all my personal and confidential documents. There are three primary reasons for choosing AWS S3: affordability, speed, and reliability. If you are working on the AWS cloud, the usage of S3 is inevitable. S3 plays a critical role in storing objects in hot and cold storage. Sometimes you need to upload a payload or file objects to S3 programmatically via your performance test script. This article will help you to upload files to AWS S3 in JMeter using Groovy.

What Is S3?

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.

S3 comes with various storage classes: S3 Standard, S3 Intelligent Tiering, S3 Glacier Instant Retrieval, and more.

Prerequisites

The following are the prerequisites for uploading files to AWS S3:

  • AWS Account
  • AWS IAM User with S3 access policy
  • Basic knowledge of AWS services
  • Basic knowledge of JMeter
  • Basic knowledge of AWS SDK (Java)
  • File(s) to upload

AWS IAM User

  • Login to your AWS account and open IAM service. Click Users under Access Management.
  • Click Add users button to set the user details and credential type as shown below.
    • Enter jmeter-s3-access in User name and check Access key - Programmatic access.
  • This user will have access to the AWS service programmatically, not from the user console. Click Next: Permissions.
  • In the Set permissions section, click Attach existing policies directly, and filter the policies of S3 by typing s3.
  • For the demonstration purpose, let us go with AmazonS3FullAccess. Check AmazonS3FullAccess and then click Next: Tags. But for the production server, follow the zero trust framework.
  • Adding tags is optional, but it is recommended to have relevant key-pair values.
  • Click Review and then click on Create user.
  • Copy the Access key ID and Secret access key to a secured location. Alternatively, you can download the .csv file.

JMeter Test Plan

By default, JMeter doesn't have the feature to upload the artifacts to AWS S3. To extend the functionality, we must leverage the JSR223 Sampler.

Here is the complete playlist of JMeter series which will help you to become a hero within one week.

JSR223 Sampler

It is not possible to write upload to the S3 code block natively. We must leverage the latest version of AWS SDK for Java. To add the AWS SDK as a dependency to the JSR223 Sampler, the easiest way is to leverage Grape. Personally, I have not tried the JMeter Maven plugin. I found Grape is simple to get started.

What Is Grape?

The Groovy Adaptable Packaging Engine or Groovy Advanced Packaging Engine, Grape, is a JAR dependency manager which is built in with Groovy.

Using the @Grab annotations in JSR223, you can add Maven repository dependencies to the classpath.

@Grab(group='software.amazon.awssdk', module='s3', version='2.17.172', scope='test')


The above annotation will download the AWS SDK S3 dependencies. To download multiple dependencies, use @Grapes annotations.

@Grapes(
    @Grab(group='software.amazon.awssdk', module='s3', version='2.17.172', scope='test'),
    @Grab(group='software.amazon.awssdk', module='sts', version='2.17.172', scope='test')

)


To change the source, use @GrabResolver annotation.

@GrabResolver(name='restlet', root='http://maven.restlet.org/')


Hello Time Using Grape in JSR223

Let us understand how Grape works in the JSR223 Sampler in JMeter by writing a simple snippet with @Grab annotations. To explain it with a simple Hello world, pardon Hello time example, let us use the below code snippet. Copy and paste the below code into your JSR223 Sampler in JMeter.

@Grapes(
    @Grab(group='joda-time', module='joda-time', version='2.10.14')
)
import org.joda.time.LocalDateTime

LocalDateTime currentDateTime = new LocalDateTime()
log.info "Local Time is " + currentDateTime


Let us slice each line. @Grapes is the optional inception annotation which has @Grab annotation. One or more @Grab annotations can be placed anywhere in the JSR223. In this example, @Grab annotation will manage the joda-time dependency and prints the current date and time in the Log Viewer. By default, it will download the dependencies from mvnrepository.com.

Click on the Run button in JMeter to see the log message in the Log Viewer. Once you click on the Run button, the above script will download the dependencies from the source and keep the JARs in /Users/<user>/.groovy/grapes in Mac, C:\Users\<user>\.groovy\grapes in Windows OS.

The dependencies download will happen only for the first execution. The eventual execution in JMeter will be fast.

AWS S3 SDK

There are multiple methods available to upload the artifacts to S3, e.g., via AWS CLI, AWS SDK, HTTP requests, and more. The official AWS SDK opens the door to building applications for AWS via its API.

By leveraging the AWS SDK for Java, it is easy to interact with the AWS services. The latest version of the AWS SDK for Java is v2. v2 is a major rewrite of v1. The v2 version is packed with a nonblocking I/O architecture using Netty where it achieves high concurrency with fewer threads. Also, it supports HTTP/2 and automatic pagination.

Upload Files to S3 in Groovy

We are going to leverage Grape in Groovy in the JSR223 Sampler by adding AWS SDK for S3. Head to https://mvnrepository.com/artifact/software.amazon.awssdk/s3 and select the latest version. At this time of writing, the latest version is 2.17.172.

Click Grape and copy the annotation as shown below.

Upload files to AWS S3 in JMeter using Groovy - Grape
Upload files to AWS S3 in JMeter using Groovy - Grape.

Then, copy and paste the below snippet into the JSR223 Sampler.

@Grapes(
    @Grab(group='software.amazon.awssdk', module='s3', version='2.17.172', scope='test')
)
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.s3.S3Client
import software.amazon.awssdk.services.s3.model.*

import java.io.File
import java.nio.file.Paths

// Configurations
String accessKey = vars.get("AWS_ACCESS_KEY")
String secretKey = vars.get("AWS_SECRET_KEY")
String bucketName = vars.get("AWS_BUCKET_NAME")
String strFilename = "C:\\temp\\result.json"

try {
	// Set region
	Region region = Region.US_EAST_1
	
	// Create credentials
	AwsBasicCredentials awsCreds = AwsBasicCredentials.create(
	      accessKey,
	      secretKey)
	
	// Build S3 Client
	S3Client s3 = S3Client.builder()
	      .region(region)
	      .credentialsProvider(StaticCredentialsProvider.create(awsCreds))
	      .build()
	
	// Create file object
	File s3Obj = new File(strFilename)
	
	// Create PUT request
	PutObjectRequest request = PutObjectRequest.builder()
	      .bucket(bucketName)
	      .key(s3Obj.getName())
	      .build()
	
	// Upload file
	s3.putObject(request, Paths.get(strFilename))
}

// Catching exception and displaying it in the Sample result
catch (S3Exception e){
	SampleResult.setSuccessful(false)
	SampleResult.setResponseMessage(e.getMessage())
	SampleResult.setResponseCode(null)
}


Let us slice the above code.

  1. After adding the import statements, the first block represents the configurations such as AWS Access, Secret Key, and Bucket name.
  2. The next configuration is the AWS region.
  3. AwsBasicCredentials block creates the credentials to access the bucket
  4. S3Client block creates an S3 client using the AwsBasicCredentials credentials.
  5. s3Obj is the file to be uploaded.
  6. PutObjectRequest will build the request.
  7. s3.putObject will upload the file by leveraging the request.

If any exceptions occur, JMeter will display the exceptions in the sampler with the exception details for troubleshooting.

The variables AWS_ACCESS_KEY, AWS_BUCKET_NAME, and AWS_SECRET_KEY are available in the Test Plan.

Here is the repository to download the sample JMeter test plan for your reference.

Never ever store the AWS credentials in the test plan. Pass the credentials via command line, environment variables, or programmatically generate them.

Congratulations! Now you know how to upload artifacts to S3 programmatically in JMeter using JSR223 Sampler.

Conclusion

Uploading artifacts to S3 is just the beginning. By leveraging the AWS SDK for Java, it is possible to interact with AWS services from JMeter using Groovy. E.g, after JMeter test execution, you can upload the results in a zip file using the above snippet. If you have any other use cases for JMeter, please let me know in the comments.

AWS Annotation Software development kit Upload Dependency Groovy (programming language)

Published at DZone with permission of NaveenKumar Namachivayam, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Why Use AWS Lambda Layers? Advantages and Considerations
  • Monitoring and Logging in Cloud Architecture With Python
  • The Essentials of Amazon S3 Glacier for Affordable and Compliant Long-Term Data Archiving
  • Is It Okay To Stop Running Your Tests After the First Failure?

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!