A Real-World Introduction to Event-Driven Architecture

What Is an Event?

An event indicates a state change: that something has happened. This ‘something’ could be a hardware sensor reading that passes a threshold, or a software event from game logic, or user input such as a keystroke or mouse click. Events are used to signal to interested parties that a state change occurred

The World Is Event-Driven

Let's use a real-world analogy to illustrate how events can be handled. Suppose you live in a shared house with a group of other people. You all like cake and one of your housemates, Bob, bakes a delicious chocolate cake every day. Bob works shifts so you can't predict when he is baking. You prefer cake fresh from the oven, but how do you know when to go to the kitchen?

You could regularly check to see if a cake is ready, but you get tired of making regular trips downstairs to ask Bob if there's cake, and there is only a tiny chance you'll choose the correct time. It would be better if Bob just came to tell you when he's finished a cake.

However, you are just one of several consumers of his cake, and notifying you all takes up his time (when he knocks on your door, you take too long to answer, and he can't move on to the next person until he's told you about the cake). Bob can't readily scale up to include the increasing number of housemates that want to know when there's cake.

Bob decides to use a publish and subscribe pattern. He rings a bell to signal that the cake is ready. You listen for the cake event, and when it happens, you decide if you want to go to the kitchen to eat the cake. Now Bob doesn't have to know anything about his cake consumers. He can bake more cakes since nobody is holding him up. He isn't interested in how many people eat his cakes, who they are, or when they eat them.

Alongside the chocolate cake, Bob starts making a fruitcake that you don't enjoy. He still rings the bell to signal a chocolate cake event, and he uses the buzzer on his kitchen timer to signal about a fruitcake. You only listen for the bell, but your fruitcake-eating housemates are tuned to the buzzer too.

Bob steps up his cake production and there are so many cake events that he asks his friend Alice to help him. Alice sets up a WhatsApp group for each cake type, which you, your housemates, and the neighbors, can join. Members receive a message whenever their preferred cake is ready.

Alice is Bob's broker: Bob uses the bell and buzzer to tell her when the cake is ready and Alice posts messages to the appropriate group as they occur. When Bob gets other bakers to help him make more cakes, she does the same for them.

Some housemates don't use WhatsApp, so Alice also writes a log of the cake events in a book, listing them in the order that they happen, the time they happened, and the type of cake event. Cake 'subscribers' may consult the logbook, scrolling back for a history of cake events, or checking the latest cake on the list.

What Is Event-Driven Architecture?

An event-driven architecture (EDA) raises and responds to events. An event producer triggers events and these reach interested consumers, as messages, via an event channel. The producer is loosely coupled or completely decoupled, from event recipients who process the messages asynchronously.

Our example above was event-driven, with Bob acting as the producer, decoupled from the cake consumers by his association with a broker, Alice. It illustrated publish and subscribe, a common architectural pattern seen in event-driven systems. An event producer (publisher) sends event messages when a state change occurs. Event subscribers then consume them as they occur and invoke their business logic in response.

When a dedicated broker sits between the publisher and subscribers, it receives all the publishers' events and notifies those who have registered interest in receiving them. The broker typically routes events to subscribers as they happen but in an event-streaming pattern, the broker also records events in an event stream. Event consumers access the stream at any time, perhaps reading the most recent message or batch processing a series of them from the last time they checked the stream.

Benefits of Event-Driven Architecture

An event-driven architecture eliminates the need for a consumer to poll for updates; it instead receives notifications whenever an event of interest occurs.

Decoupling the event producer and consumer components is also advantageous to scalability because it separates the communication logic and business logic. A publisher can avoid bottlenecks and remain unaffected if its subscribers go offline, or if their consumption slows down. If any subscriber has trouble keeping up with the rate of events, the event stream records them for future retrieval. The publisher can continue to pump out notifications without throughput limitations and with high resilience to failure.

Using a broker means that a publisher does not know its subscribers and is unaffected if the number of interested parties scales up. Publishing to the broker offers the event producer the opportunity to deliver notifications to a range of consumers across different devices and platforms.

Estimates suggest that 30% of all global data consumed by 2025 will result from information exchange in real-time. EDA slots alongside a traditional architecture and provides the functionality required to meet the demands of real-time digital interactions. It has tremendous potential to serve the growing demand from use cases that need to process data immediately to support a sophisticated user experience (within online messaging, financial systems, and informational services, to name a few).

Challenges of Building a Real-Time Event-Driven Architecture

Using an EDA gives an opportunity to offer real-time updates across a range of applications. But for any business, building the real-time capabilities to deliver these experiences at scale is risky, complex, costly, and time-consuming for a number of reasons including:

Technical debt: Developing the feature-complete technology that is required to guarantee bidirectional, low-latency real-time digital experiences is complex and time-consuming. Organizations spend months stitching together disparate systems (e.g for WebSockets, low latency traffic routing, scaling) to build basic real-time capabilities. This often results in technical debt and additional engineering investment for must-have features.

Performance: Poor latency and bandwidth concerns create uncertainty when it comes to designing, building, and scaling real-time features.

Integrity: Many applications rely on a sequence of messages that mutually depend on each other. However, if these messages are lost or unordered, your company is unable to deliver the seamless real-time digital experiences users expect. To illustrate with our baking example: if Bob's messages are lost, Bob's customers miss out on chocolate cake! The engineering required to deliver real-time digital experiences with data integrity is truly complex and often results in unacceptable tradeoffs just to move forward - companies end up sacrificing performance at scale for data integrity.

Reliability: Designing, building, and operating your own globally-distributed fault-tolerant real-time infrastructure is complex and costly. Even the level of reliability provided by some of the latest technology companies is insufficient.

Scalability: When organizations start to scale, they can pay the price for earlier choices. Localized, 'good enough' infrastructure is unable to scale or provide the elasticity needed to meet customers' real-time requirements. Unless your company is very good at automation and DevOps, then it can be highly problematic, and using a robust service is a far better option than running your own.

Click here to read the longer version of this article.