Messaging in AWS Using SNS and SQS
Learn how to use asynchronous messaging in AWS using SNS and SQS.
Join the DZone community and get the full member experience.Join For Free
There are two main communication paradigms in event-driven architectures used in microservices design.
- Queueing provides a messaging system to integrate two different services.
- Publish/subscribe messaging is a form of asynchronous service-to-service communication used in microservices architectures. In contrast to queueing, publish/subscribe messaging allows multiple consumers to receive each message in a topic.
They allow us to de-couple producers and consumers of messages. By combining publish/subscribe messaging systems with queueing systems, we can build fault-tolerant, scalable, resilient, and reactive application architectures. Amazon Web Services (AWS) offers a number of services which provide these two communication paradigms. In this article, we will learn how to program AWS services – Simple Notification Service (SNS) for publish/subscribe messaging and Simple Queue Service (SQS) for queueing using AWS SDK in Java.
We have an application which needs to publish notifications if there is a change. This change is published to a topic in SNS. There are different subscription mechanisms available for an SNS topic – SQS being one of them. We will use an SQS queue to receive change notifications. An application may poll SQS queues for any message.
To develop and deploy the system, we'll need following infrastructure components:
- SNS topic.
- SQS queue.
- Subscription to the topic.
- Permission for SNS to publish to the SQS queue.
We will use Terraform to provision the infrastructure on AWS. Terraform is an open-source infrastructure as code software tool created by HashiCorp.
Let's create the SNS topic. To create an SNS topic we only need to provide a name.
We need permission to publish to this SNS topic. In our architecture, an application will publish change notifications to the topic. The application will use a user account with permissions to publish to the topic.
We have to provide a name for the queue and a policy which allows SNS to send messages to the queue.
Next, create a subscription of the SNS topic for the SQS queue.
Next, allow SNS to publish messages to the queue.
Creating the Infrastructure
Let us run our Terraform scripts to create the infrastructure.
The output of
terraform apply provides us the SQS Queue URL and the SNS topic ARN that we will use in our application.
Develop the Application
Next, we develop our application that will publish a notification to an SNS topic and also poll an SQS queue to receive messages. We will use Java and the AWS SDK to develop our application.
Send Notifications to an SNS Topic
Receive Messages From an SQS Queue
The complete code is available in my GitHub repo. Run the application as a Spring Boot application and hit the application URL to send a message. The output shows the received message from the SQS Queue.
In this article, we reviewed two important communication mechanisms in event-driven programming. We implemented an architecture on AWS leveraging the services – SNS and SQS – that implement these communication mechanisms. We provided a sample Java application using the AWS SDK to send and receive messages from our system.
Opinions expressed by DZone contributors are their own.