Getting Started With Amazon SQS

Like any other queue, SQS provides a way for the asynchronous integration of two components of a system. As a result, it responds faster to the requests and decouples the component so it is easier to manage and scale. Apart from these queue characteristics, being part of the AWS, it provides other benefits like scalability, reliability, and a fully managed service — meaning that the headache of maintenance is off-loaded to AWS rather than us.

For more details on various features of the SQS, please check the nice documentation from AWS. This article provides a quick guide on how to use an SQS with a simple Java program to send the message to SQS from a producer running in a process and consume the same message by a consumer running in another process.

Prerequisites

In order to use SQS, the following is needed.

Create an SQS

I'm assuming that you have an AWS account and have created an SQS for yourself. If not, please check the documentation, which details very well how to create one. Each SQS created in AWS has a unique URL associated with it that is used to access it.

The format of this URL is https://sqs.<region>.amazonaws.com/<account_id>/<queue_name>. Please take a note of this URL, as we will need it in our code. You can see the URL for your SQS in the details screen of SQS in AWS console. Please see Figure 1 for reference.

For this article, I have created an SQS with the name sqs-demo in my AWS account, which we are going to use in the code.

Figure 1.

AWS SDK

You have Java AWS SDK in your system. Please check the AWS site to download it.

Access Permission

In order to connect to SQS from your system through an external system, you need to have permission to do so. For that, we need to sign the calls that we make to SQS with an Access Key ID and a Secret Access Key. If we are using SDK, it is signed automatically under the hood by SDK — but it expects these credentials to be available at designated places to be read from. There are various ways of doing it, but that is long enough to be a topic for another article. For simplicity, we choose one of them which is creating the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_KEY with their values.

You can find access keys from the My Security Credentials section of the AWS console. But you can only see the aws_secret_access_key only once when they are created. If you have not saved your aws_secret_access_key, then you need to generate a new access key, which would give you the credential keys.  Please see Figure 2 to see where to get the credentials in AWS console.

Figure 2.

Code

Now, when we are ready with all the prerequisites, let us see how the code is organized.

Each message that we put in SQS has a body. If needed, you may also put some metadata with it. In this app, we are associating two metadata fields: a string type and a number type.

The application demonstrates the following:

• Creation of message with a string metadata and a number metadata.
• Putting it in SQS.
• Picking up these messages from SQS.
• Read the content of messages along with metadata.

To accomplish this, the following classes are created.

SQSMessageScheduler

This class interacts with SQS to put the message into SQS, using a reference of AmazaonSQSClient. It has only one method, scheduleMessage, which does the following:

• Creates the body of the message to be put in SQS.
• Creates metadata to be set into the message. Metadata is set in the form of a hash map with a user defined ID as key and MessageAttributeValue as value. MessageAttributeValue is the class that you will find in AWS SDK. This map would be set in the request in the subsequent steps. We are storing two metadata fields in this map.
• Creates a SendMessageRequest. 
• Sets the body and metadata map into this request.
• Associates the queue URL with this request.
• Uses the AmazaonSQSClient instance to send the message as a main class for the producer process.

Producer

This class is for illustrating sending a number of messages to SQS. We create a single thread executor that is scheduled to run for five times wherein each time, it creates a new body, string, and number metadata and then invokes the method scheduleMessage from SQSMessageScheduler to put these messages in SQS.

SQSMessageConsumer

This class interacts with SQS to read the message from SQS using a reference of AmazaonSQSClient. It has only one method, receiveMessage, which does the following:

• Invokes receiveMessage on AmazaonSQSClient to receive all the messages that are in the SQS.
• For each of these messages, it reads the body part.
• Reads the metadata associated with the message from the AttributeValue map using the IDs that were set when the messages were put in SQS.
• After processing the message, it deletes the message from SQS, using it as main class for the Consumer process.

Consumer

In order to read messages from SQS, we need to keep polling into it. This class implements the polling behavior with a single thread executor which is scheduled to run five times and invoke the receiveMessage method from SQSMessageConsumer to process the messages read.

import java.util.HashMap;
import java.util.Map;

import com.amazonaws.services.sqs.AmazonSQSClient;
import com.amazonaws.services.sqs.model.MessageAttributeValue;
import com.amazonaws.services.sqs.model.SendMessageRequest;
import com.amazonaws.services.sqs.model.SendMessageResult;


/**
 * Class which interacts with SQS and schedules message into it
 * 
 */
public class SQSMessageScheduler {

 private String sqsQueue;
 private AmazonSQSClient client;

 public SQSMessageScheduler(String sqsQueue) {
  super();
  this.sqsQueue = sqsQueue;
  client = new AmazonSQSClient();
 }


 /**
  * Schedules a message to SQS
  * @param body - text of the message
  * @param strMetadata - a string meta data for the message
  * @param intMetadata - a number meta data for the message
  * @return
  */
 public boolean scheduleMessage(String body, String strMetadata, int intMetadata) {
  System.out.println("scheduleMessage:" + body);
  try {
   //Map for the Attributes which will hold the meta-data for our message
   Map < String, MessageAttributeValue > messageAttributes =
    new HashMap < String, MessageAttributeValue > ();
   //Put string metadata
   messageAttributes.put(
    "ID_FOR_STRING_METADATA",
    new MessageAttributeValue().withDataType("String").withStringValue(strMetadata));
   //Put number metadata 
   messageAttributes.put(
    "ID_FOR_NUMBER_METADATA",
    new MessageAttributeValue().withDataType("Number").withStringValue(Integer.toString(intMetadata)));

   //Create the request setting the message text and attributes
   SendMessageRequest request = new SendMessageRequest();
   request.withMessageBody(body);
   request.withQueueUrl(sqsQueue);
   request.withMessageAttributes(messageAttributes);

   //and send the message to SQS
   SendMessageResult result = client.sendMessage(request);

   System.out.println("Request scheduled " + request + ", with the result:" + result);

   //Successfully sent the message
   return true;
  } catch (Exception e) {
   e.printStackTrace();
   //Faliure in sending message 
   return false;
  }
 }

} -

import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;


public class Consumer {

//Replace account_id with actual AWS account id
private static SQSMessageConsumersqsMsgConsumer = 
new SQSMessageConsumer("https://sqs.ap-southeast-1.amazonaws.com/<account_id>/demo-sqs");

//How many times we want to do polling on SQS
private static int count = 0;

/**
 * This method receives messages SQS in a single threaded thread-pool
 * @param iterations - count for which SQS is checked for messages
 */
public static void startReceiving(final int iterations){
//Create a single threaded thread-pool
final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);

//Implement the runnable method which checks for message from SQS on every invoke
final Runnable consumeTask = new Runnable(){
public void run(){
// Do till the count is not over
if (count > iterations){
System.out.println("Exiting...");
System.exit(0);
}
//Receive the message from SQS
sqsMsgConsumer.receiveMessge();
count++;
}//run()
};//Runnable

//Schedule the thread to do the task every 2 sec
scheduler.scheduleAtFixedRate(consumeTask, 2, 2, TimeUnit.SECONDS);
}


public static void main(String[] args){
System.out.println("Consumer ready to consume...");
startReceiving(5);
}



}
-

Maven POM File

To make it easier, here is the Maven POM file that I have used to build the JAR file. This will create the self-sufficient JAR file SQS-Demo-1.0-jar-with-dependencies.jar with all the dependencies bundled into it.

<project
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>SQS-Demo</groupId>
<artifactId>SQS-Demo</artifactId>
<version>1.0</version>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<!-- any other plugins -->
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.22</version>
</dependency>
</dependencies>
</project>

Executing the Code

Now, let's get into executing the code.

Producer

Open up a command prompt, navigate to the folder where you have the SQS-Demo-1.0-jar-with-dependencies.jar created and run the following command: java -cp SQS-Demo-1.0-jar-with-dependencies.jar Producer.

This will show that five messages are sent to SQS in the console. The screenshot of my console is shown in Figure 3:

Figure 3.

You may see the messages in the SQS from AWS console, as well. Log into AWS, navigate to the SQS Service section, select the queue, and select View Messages. The screen that pops up will ask you to start polling, as seen in Figure 4. Remember that entities interested in messages in SQS need to do a polling on SQS. Click Start Polling and you will see all the messages that have been sent by the Producer, as seen in Figure 5.

Figure 4.

Figure 5.

Consumer

Open up another command prompt, navigate to the folder where you have the SQS-Demo-1.0-jar-with-dependencies.jar created and run the following command: java -cp SQS-Demo-1.0-jar-with-dependencies.jar Consumer.

This will show that five messages are received from SQS in the console. The screenshot of my console is shown in Figure 6.

Conclusion

This article gives a brief introduction of how to use SQS with Java SDK by illustrating how to put a message in SQS as well as how to read it. This may give a good start for those who want to deep-dive into SQS. The article just touches upon the SQS and how to use it; however, in order to take full advantage of SQS, we need to be aware of its basic architecture and the other features that it comes with. Going through the developer guide provided by AWS on its website may be a good idea for learnubg nire about SQS.