Where Do I Start With Event-Driven Architecture?

Jonas Bonér, Founder and CTO of Lightbend, inventor of the Akka project, co-author of the Reactive Manifesto and a Java Champion, shared his thoughts on the state of event-driven architecture in advance of speaking at the Reactive Summit in Austin, Texas on October 18 on How Events Are Shaping Modern Systems. Click here to register.

What do you see as the most important elements of developing event-driven architecture?

It starts with the design and understanding the problem since no architecture, tools, or products can help you fix a bad design.

To be successful in modeling event-driven systems ones has to take a different approach than traditional domain modeling. In Domain-Driven Design (DDD) people are used to starting with the domain objects, the nouns in the domain. The problem with this is it puts the focus on structure too early in the design process. It is often better to start trying to understand what happens in the system, the verbs. That will help you to understand the events that drive the business logic, the workflow.

A great way to do that is to use Event Storming, in which you put all the stakeholders in the same room with a bunch of post-it notes, to try to figure out how information flows in the system — the events — what initiates them — the commands — and their effects. Then let these insights drive the definition of the domain models, Bounded Contexts, Aggregates, and their protocols.

Once you have an event-driven design it's time to pick the right patterns and architecture. For example consider using an Event Stream for communication and event notification between services, rely on Event Sourcing and CQRS for event-based persistence, or using Stream Processing to get real-time insights into the event streams, as they enter the system.

It can be helpful to not build the infrastructure from scratch but rely on existing platforms and tools. In my experience, basing the architecture on a Reactive foundation is paramount, since it can do much of the heavy lifting in terms of elasticity and resilience.

How is your company using event-driven architecture to provide solutions to clients?

Our platform, the Lightbend Reactive Platform, is an enabler for modern event-driven architectures. It is built on the Akka reactive middleware that I created in 2009. Akka — an implementation of the Actor Model, implementing the Reactive principles — works as a distributed systems fabric providing the foundation of scalability and resilience to the rest of the platform. On its foundation, we have built other products, like the Play Framework, the Lagom framework for Reactive microservices, the newly released Lightbend Fast Data Platform.

Which programming languages, frameworks, and tools do you, or your company use, to create event-driven architecture?

We have built our Reactive platform and tools using the Scala programming language (that is developed by Lightbend). Scala is a perfect language for building these type of systems, both as an implementer of the frameworks, and as a user of them. For example, Kafka, Spark, and Flink are all built in Scala. Scala is also often the best way to use these products since it lends itself very well to an event-driven programming model with its Functional Programming side — providing first-class functions, working with immutable values, and a rich set of combinators (map, filter, fold, groupBy, and the likes) for transforming and composing events as they flow downstream.

How has the development and deployment of event-driven architecture changed?

The idea of Event-driven Architecture is, at the very least, 15-20 years old. During the years it has gone through several hypes and backlashes, the most recent one being SOA. Now it’s hyped again, and some people ask if microservices is not just SOA dressed up in new clothes. The answer is both yes and no. Yes, because of the initial goals—decoupling, isolation, composition, integration, discrete and autonomous services—are the same. And no, because the fundamental ideas of SOA were most often misunderstood and misused, resulting in complicated systems where an Enterprise Service Bus was used to hook up multiple monoliths, communicating over complicated, inefficient and inflexible protocols.

The world of the software architect looks very different today than it did 10-15 years ago when SOA emerged. Today, multi-core processors, cloud computing, container-based deployment models and orchestration, mobile devices and the IoT are emerging rapidly, which means that all systems are distributed systems from day one. This is a vastly different and more challenging world to operate in, but also a world where the ideas of event-driven architecture and event-driven design are more relevant than ever.

What are the most common issues you see affecting the development and deployment of event-driven architecture?

The biggest hurdles to success in my experience are preconceived ideas and expectations, old habits and ways of thinking, and clinging onto favorite tools and techniques. Perhaps more generally: a reluctance to challenge status quo and do the hard work of unlearning. Past that, you are up for a whole new world of challenges, most of them quite fun and interesting though.

Do you have any concerns regarding the current state of event-driven architecture?

We have come a long way. The recent developments in infrastructure and tools, and growing understanding and appreciation of distributed systems and the hard-won wisdom by people like Pat Helland, Leslie Lamport, and Jim Gray, have put us up for success.

But I think we still have a long way to go as an industry. In particular when it comes to understanding and analyzing the runtime behavior of large-scale, event-driven, distributed, and asynchronous systems. Traditional monitoring and logging techniques usually don’t cut it in this new world. It calls for different approaches, and for us to embrace the fact that what we are building are often non-deterministic and adaptive complex systems of autonomous services, where feedback loops — positive and negative — play a huge role.