Event-Driven Fractals

Message-passing applications are one of the main components of reliable distributed systems; above all, they make it feasible to decouple the "when" and "where" of a problem from "who does it how." Event-driven applications go one step further and give you the causal chain of your system as a first-class citizen.

While the difference between an event-driven system and one that’s not event-driven is pretty clear and obvious to most software engineers, there are a gazillion ways to design such a system. Each has its own trade-offs and is useful in a specific context. The one I’m going to talk about in this article is modeling applications as state machines that compose and form a fractal-like structure — as each component is an application, and composed ones are also the same kind of application.

For doing that, I’ll use Edomata as an example, a purely functional library in Scala that I wrote with this idea. But the main idea is general and can be implemented in any language with idiomatic considerations in the host language.

Why?

Besides their intuitive and easy-to-understand nature, state machines are one of the building blocks of our understanding of systems in general, and have been studied widely in several fields such as mathematics, distributed systems, hardware design, and software engineering, to name a few.

This feature comes to help us in expressing our ideas and models eloquently, while preserving its links to the real world so that we can implement it easily. Moreover, the composition feature helps us to divide and conquer the problem. We can understand a small state machine; we can understand how its composition behaves — so we can build larger state machines that we can understand, but couldn't have understood easily otherwise.

Event-Driven State Machines

Generally speaking, state machines consist of these elements:

• State: Possible states (obviously)
• Input: Possible input
• Initial state
• Transition: A function that determines how a state machine changes its state

A state machine is event-driven if the transition from one state to another is triggered by an event.

For example, if we define our transition function as fold-over events, we’ll get an event-sourced application! On the other hand, if we emit events between transitions, those events can be used for integration with other state machines! And that’s the idea of event-driven architecture.

Primitive State Machines

Let’s see how one of the primitive state machines in Edomata works. It’s named Decision and models pure event-sourced programs. It works as follows:

• Initially, it is indecisive and contains a result value.
• It can decide on some logic, then accept or reject.
• If accepted, it must have one or more events (as it models event-sourced applications) and a result value.
• If rejected, it must have one or more errors.

How Does It Compose?

A Decision contains a result value if it’s not rejected, so we can define a .map function that takes a function and returns a new Decision whose result value is the result of mapping using the provided function. The resulting Decision is also a Decision obviously, which is similar to how we started.

We can also bind two Decisions together. So we have Decision A that may contain a result value a, and we can provide a function that takes a and returns a new Decision B. We get a new Decision that is the result of composition and behaves like the following:

This shows that errors cause short circuits, and also, events do accumulate.

The interesting property is its self-similar structure (like a fractal) that lets us encapsulate some logic behind a state machine, which may be the result of composition of several other state machines.

But it does not end here.

Compound State Machines

That was an example of a simple primitive state machine. State machines can also embed other state machines and add new behaviors on top of other state machines. That’s what an Edomaton does, for example; it models general event-sourced applications and can do the following:

• Read state
• Run an idempotent side effect
• Decide (using Decision)
• And publish external events (note that in well-designed systems, event-sourcing is an internal implementation detail and is hidden from outside systems, and journal is not meant for integration; that’s why there’s a separate event channel here)

How Does It Compose?

It behaves exactly like Decision, but has these new capabilities:

• External events are also accumulated if not rejected
• External events get reset, when rejected
• Also, it can recover a rejected decision by publishing external events, so it can be used as some form of compensation event

Again, we see a self-similar structure with all the properties we’ve talked about before. Programs written this way are truly fractal-like structures, created from little elements that are all the same fractal!

In Action

The following is an example from the Edomata tutorial. As you can see, it’s a very simple code that manipulates data in memory. No magic or supernatural phenomena found in most of the frameworks! (But hey, that’s a library.)

enum Account {
  case New
  case Open(balance: BigDecimal)
  case Close
  
  def open : Decision[Rejection, Event, Open] = this.decide { 
    case New => Decision.accept(Event.Opened)
    case _ => Decision.reject(Rejection.ExistingAccount)
  }.validate(_.mustBeOpen)
  
  def close : Decision[Rejection, Event, Account] = 
    this.perform(mustBeOpen.toDecision.flatMap { account =>
      if account.balance == 0 then Event.Closed.accept
      else Decision.reject(Rejection.NotSettled)
    })
  
  def withdraw(amount: BigDecimal): Decision[Rejection, Event, Open] = 
    this.perform(mustBeOpen.toDecision.flatMap { account => 
      if account.balance >= amount && amount > 0 
      then Decision.accept(Event.Withdrawn(amount))
      else Decision.reject(Rejection.InsufficientBalance) 
    }).validate(_.mustBeOpen)

  def deposit(amount: BigDecimal): Decision[Rejection, Event, Open] = 
    this.perform(mustBeOpen.toDecision.flatMap { account =>
      if amount > 0 then Decision.accept(Event.Deposited(amount))
      else Decision.reject(Rejection.BadRequest)
    }).validate(_.mustBeOpen)
  
  private def mustBeOpen : ValidatedNec[Rejection, Open] = this match { 
    case o@Open(_) => o.validNec
    case New => Rejection.NoSuchAccount.invalidNec
    case Close => Rejection.AlreadyClosed.invalidNec
  }
}

Testing

One of the nicest properties of state machines we’ve not talked about yet is that state machines are not the machine — they are the definition of the machine! So we can run a state machine in any state easily, and that’s a high win in testing. It allows us to easily use property testing on our business logic, which shows how high it can reach in level of testing.

Account.New.open
// res1: Decision[Rejection, Event, Open] = Accepted(Chain(Opened),Open(0))
Account.Open(10).deposit(2)
// res2: Decision[Rejection, Event, Open] = Accepted(Chain(Deposited(2)),Open(12))
Account.Open(5).close
// res3: Decision[Rejection, Event, Account] = Rejected(Chain(NotSettled))
Account.New.open.flatMap(_.close)
// res4: Decision[Rejection, Event, Account] = Accepted(Chain(Opened, Closed),Close)

Running

Running such a system in production is trivial too. The only thing required is a back end capable of running state machines. It shows how much our business logic is decoupled from infrastructure concerns and implementation, which is also a high win!

val pool : Resource[IO, Session[IO]] = ??? // postgres connection pool

val buildBackend = Backend
  .builder(AccountService)
  .use(SkunkDriver("domainname", pool))
  .persistedSnapshot(maxInMem = 200)
  .withRetryConfig(retryInitialDelay = 2.seconds)
  .build


// Now you have a production ready system!

val application = buildBackend.use { backend =>
  val service = backend.compile(app)
  // compiling your application will give you a function
  // that takes a messages and does everything required,
  // and returns the result.

  // Now we can send a command to our service!
  service(
    CommandMessage("cmd id", Instant.now, "aggregate id", "payload")
  ).flatMap(IO.println)
}

Other Examples

There are other kinds of state machines implemented in Edomata, such as ResponseT (which models an event-driven program response), Stomaton (which models raw event-driven state machines, used in a CQRS style), Action (impure event-sourced applications), and a few others.
For more examples and detailed content, visit the project’s website and docs.

Also feel free to start discussions or submit issues on GitHub.

Conclusion

You might say: Well, this is the definition of monad. What’s new?

I would say yes! But calling it a monad wouldn’t make it more useful to people, would it? Most software engineers and developers today are not category theory experts, nor do they want to be so.

But this doesn’t mean they shouldn’t benefit from the plethora of useful and extremely helpful ideas found in these topics in action. Edomata is a library designed with this mindset; it can help you develop your event-sourced or CQRS applications rapidly, and it lifts the burden of unneeded complexity for you. It is extremely modular and can be made to work as you wish, as it’s a library, not a framework.

It can also be a good example of an un-opinionated design for these kinds of systems that is portable to other ecosystems too.

I would like to know if you find this interesting or have any kind of questions or feedback. Let me know in the comments, or let's have a discussion on GitHub!