Thanks to its internal architecture based on routers and brokers, EnMasse provides two different messaging mechanisms: direct messaging and store and forward.
The direct messaging mechanism is not new. AMQP 1.0 is a peer-to-peer protocol, so you can have two clients directly connected each other. In this way, the producer is able to send a message only when the consumer is online (providing “credits”), and it receives an acknowledgement, which means that the consumer has received the message. Between the two parties, there is a single contract so that the producer knows that the message is received by the consumer when it gets feedback.
EnMasse provides this kind of mechanism in a more reliable and scalable way through the router network, where routers are connected to each other making a “mesh" — the clients aren’t connected directly but instead through this network. Every router, unlike a broker, doesn’t take ownership of the message but just forwards it to the next stop in the network in order to reach its destination. If a router goes offline, the network is automatically reconfigured in order to identify a new path for reaching the consumer; it means that high availability is provided in terms of path redundancy. Furthermore, as it happens in a real direct connection with AMQP 1.0, a producer doesn’t get “credits” from a router for sending messages if the consumer isn’t online or can’t process more messages. Finally, direct messaging is synchronous by nature, so it’s really useful for RPC communication.
The entity used for identifying how producers and consumers exchange messages is the address, as it’s defined by the AMQP 1.0 specification as just a string.
Figure 4: Direct messaging.
The store and forward mechanism is provided by the brokers behind the router's network in two different steps. First of all, a broker takes ownership of the received message, storing it internally (just in memory or in a persistent way), but it doesn’t mean that such a message is immediately forwarded to the final consumer, which could be offline in that moment. It allows asynchronous communication and time decoupling because the consumer can get the message later and at its own pace, which can be different from the producer. There is always a double contract between producer-broker and broker-consumer, so the producer knows that the message reached the broker, but not the consumer (a new message exchange on the opposite direction is needed for having acknowledgement from the consumer).
The entities used for storing messages are queues and topics, which allow point-to-point (or competing consumers) and publish-subscribe patterns. In any case, the name of a queue or a topic is just a string, like an AMQP 1.0 address.
Figure 5: Store and forward.
From a client's perspective, when they connect to EnMasse through the unique entry point (which is the router's network), they just connect to an address for exchanging messages; they don’t know that a broker could be behind such an address with a corresponding queue or topic.
The EnMasse operator has to define the addresses and their semantics in order to support the different messaging mechanisms as described above. Using the related console, it’s really simple and it’s a matter of defining what kind of pattern the clients need. The supported address types are the following:
- Queue: Backed by a broker for “store and forward” and providing point-to-point (competing consumer) patterns.
- Topic: Backed by a broker for “store and forward” and providing publish/subscribe patterns.
- Anycast: Similar to a queue in terms of direct messaging. A producer can send messages to such an address only when one or more consumers are listening on it and the router's network will deliver them in a competing consumer fashion (from a round-robin way to a more sophisticated one based on load balancing).
- Multicast: Similar to a topic in terms of direct messaging, it has a producer publish messages to more consumers listening on the same address so that all of them receive the same message.
In an IoT-specific use case, the main two communication patterns, like telemetry, command, or control, could be implemented in a few different ways.
In the telemetry scenario, a device could be enabled to send telemetry data only if a backend service is online and able to get such data; we don’t care about data if no one is able to process it, and we are not interested in storing the related messages. In this case, using direct messaging is the right solution. On the other side, it’s possible that we want to ingest data from devices even when no services are running (or maybe they are busy) for processing. In this case, store and forward is the way to go, putting messages inside queues or topics (depending on whether we want to distribute messages to one or more services in parallel).
In the command and control scenario, we could have the same two approaches. In one case, we want to send a command to a device only if it’s online and we are sure that it can execute (at least receive) the command itself in that moment. Direct messaging can help with this use case. On the other side, it’s possible that we want to handle situations in which the device isn’t online, but we want it to execute the command when it comes back online. In this case, the command message needs to be stored for later delivery. In order to avoid sending “stale” commands to a device that comes back online too late (for the command), the message can have a related TTL (time to leave) so that it disappears from the queue on expiration.