Google Cloud Pub/Sub - Overview
A high-level overview of event-driven microservice architecture using Google Cloud Pub/Sub - fully managed real-time messaging service.
Join the DZone community and get the full member experience.Join For Free
The Google Cloud Pub/Sub is a fully managed real-time messaging service and it helps our applications/services to send and receive messages independently between them. It helps us to build robust and scalable applications by integrating them asynchronously. It provides scalability, resilience and handles millions of messages simultaneously.
Why Google Cloud Pub/Sub?
Google Cloud Pub/Sub can be used for a lot of use cases. In general, If you want to process large amounts of data for analytics or do you want to simplify the event-driven microservices development?, then the Google Cloud Pub-Sub is the right choice.
A Use Case Behind the Google Cloud Pub/Sub
Alice and Bob working on the same project and both of them are in different parts of the world. Alice living in California, USA and, Bob living in Sydney, Australia. Alice will complete her work and assign the review to Bob and, Bob also does the same. This allows both of them to work on their working hours and no need for both of them to be available online at the same time. But project work is done. The Google Cloud Pub-Sub is following the same approach for our applications and services.
Microservice Architecture of Shoe-Mart
Assume the Shoe-Mart is a company selling various brands of shoes and shipping them to different countries for customers. When a customer orders shoes and the Shoe-Mart package the order, shipping to the customer and sending notification to the customer about the order status.
To build a simple service-oriented architecture system, we need a minimum of four microservices.
- The Package service is a service to package the customer orders.
- The Shipping service is a service to ship the ordered packages to the customers.
- The Notification service is a service to send the order status notification to the customer.
- Order service is a tied-up service for managing above all.
Now comes the implementation.
- The Order service will receive an order from the customer.
- The Order service sends the message to the packaging service (Hey! I received this NEW order).
- The package service will process the order for packing send a message to the shipping service.
- The shipping service will ship the ordered package and send a message to the notification service to notify the order shipped status to the customer.
- Similarly, the package service will send a message to the notification service to notify the customer about the updated order packaging status.
- Also, the order service will send a message to the notification service to notify the customer about the updated order status.
If Package Service is down, how to handle the requests in this architecture?
If Package Service is down and unavailable to respond to the other services, then we can implement a retry mechanism to keep or check packaging service is alive or not. But it could be a lot of work on the developer side and adding retry logic for all services and managing is quite difficult.
Introducing New Service
If we introduce a new service, then how our system looks?
Introducing a new service called "Monitoring Service". The monitoring service will log all these transactions in the central store and monitor all the services. This service will communicate with all other services, which require a change in all other services too. The development becomes hard and maintaining the system could be complicated.
If you see the above system, It may be a complicated design and there are many arrows between the services.
Think About Testing!
We need a mock service to test all services individually. Each service is dependent on another service. If you want to test a Packaging service, you can write a mock test. But, you cannot write the mock test only for the packaging service. You need to write the mock test for all other dependent services also.
Think About Scaling!
If you want to increase the instances of packaging service, you need to deploy the load balancer. If you need to cancel the order, you wouldn’t know what state of that order and it can be canceled or not.
Now, Its Time To Introduce a Google Cloud Pub/Sub
The Google Cloud Pub/Sub solves the above complications. It works as a fully-managed real-time messaging service and it helps our services to send and receive messages independently between them.
After the event is published as a message in cloud Pub-Sub, it becomes the job of the service to ensure all the systems react to the event and get it.
In general, the asynchronous integrations react to the events represented as messages. The event is represented in the real-time world as orders received, packages shipped, etc.
Let's go to the Shoe-Mart use case with Google Cloud Pub/Sub.
- The Order service will receive an order and publish an event represented as a message in Pub-Sub instead of sending a message directly to the packaging service.
- The Packaging service and Notification service will receive an event by either push or pull operation.
- The Packaging service publishes an event in Pub-Sub instead of sending a message to the Shipping service.
- Then, the Shipping service and Notification service will react to the event published by the Package service.
- The packaging service is not required to know which service is receiving this event.
- Similarly, the Shipping service publishes an event in Pub-Sub and, the Order and Notification Services will react to that event by either push or pull operation.
By using the Google Cloud Pub-Sub, we have solved the problem of the earlier architecture. All services are working independently and, each service does not require to know about other services and does not require to know about which service receives the event. All services should subscribe to an event in the Cloud Pub-Sub and receives the right message and process.
Introducing New Service in Cloud Pub/Sub
Introducing a new monitoring service with Cloud Pub/Sub to logs all transactions and store them in the central store and monitoring all the services.
It is also easier to introduce the new service called Monitoring Service. The new service should subscribe to the events from the existing cloud Pub-Sub to receive the messages from other services.
If the Package service is down, how to handle it with Google Cloud Pub/Sub?
Any service is break-down, the Cloud Pub-Sub store the messages for seven days. If the service becomes alive, then automatically, the service will receive events from the cloud Pub-Sub.
Uses of Implementing the Google Cloud Pub/Sub
- It reduces the service dependencies.
- Services can change independently and, they no need to know about the other service changes.
- The architecture with the Google Cloud Pub-Sub is more flexible.
- We do not require to worry about a single point of failure.
- It provides fully managed scalability and resilience support.
- Testing becomes easy by writing a mock test independently for each service.
Happy Architecture! Happy Coding!
Opinions expressed by DZone contributors are their own.