When Feature Flags Become Workflows: Configuring Multi-Tenant State Machines

An enrollment deadline changes for one employer. Another employer adds a documentation requirement. A third needs employees with unfinished applications to receive a different experience from employees who have never started.

Each request sounds like configuration. Yet implementing it often requires editing conditionals across backend services, notification handlers, and frontend components. The settings live outside the application, but the code that connects them remains scattered.

A health insurance marketplace illustrates this engineering problem particularly well. Employer offerings, geographic eligibility, carrier responses, and enrollment windows interact across a lifecycle that can last weeks. Modeling that lifecycle explicitly can move supported workflow variations into configuration while keeping the application responsible for reliable execution.

Recognizing When Configuration Describes Behavior

Feature flags answer questions such as whether a plan category should appear. Configuration values describe enrollment dates or reminder intervals. Neither, by itself, defines what happens when an application awaiting approval receives a request for additional documents.

Developers must supply those semantics somewhere. Without an explicit lifecycle model, they usually emerge as combinations of booleans:

if application.submitted and not application.approved:
    if application.documents_requested:
        show_document_upload()
    else:
        show_pending_review()

This works until more conditions arrive. Documents can expire, applications can be denied, and employees can submit appeals. Different services may interpret the same flags differently, producing contradictory behavior.

The useful distinction is between configurable facts and configurable transitions. Facts describe the circumstances. Transitions describe which changes are permitted given the current state, an incoming event, and those circumstances.

A rules engine can evaluate eligibility. A state machine can determine what an eligibility decision allows an application to do next. These mechanisms can coexist.

Model the Lifecycle at the Right Boundary

Start by identifying the object whose progress needs coordination. An employee account has activation states, but an enrollment application has submission and carrier-review states. Keeping those boundaries explicit becomes important when one employee can hold multiple applications.

Within an application workflow, useful states might include draft, submitted, pending_documents, approved, and denied. Events describe occurrences such as application_submitted, documents_requested, or carrier_approved.

Guards constrain transitions. A submission event might require an open enrollment window and a completed application. The event expresses intent; the guard checks whether that intent is currently allowed.

Hierarchical states help organize related behavior. An application parent can contain its review states, allowing shared handling at the parent level. SCXML provides standardized XML notation for states, nested states, events, guards, and transitions. See the W3C SCXML specification.

The following minimal model captures a documentation loop:

<scxml xmlns="http://www.w3.org/2005/07/scxml"
       version="1.0" initial="submitted">
  <state id="submitted">
    <transition event="documents_requested"
                target="pending_documents"></transition>
    <transition event="carrier_approved"
                target="approved"></transition>
    <transition event="carrier_denied"
                target="denied"></transition>
  </state>

  <state id="pending_documents">
    <transition event="documents_uploaded"
                target="submitted"></transition>
  </state>

  <final id="approved"></final>
  <final id="denied"></final>
</scxml>

This example deliberately ends at the review decision. Supporting appeals or policy activation would require extending the model. Making the boundary explicit prevents a terminal state from accidentally representing an unfinished business process.

Separate Definitions From Execution

The configuration should describe permitted behavior. The execution layer should enforce it consistently.

An event-processing service loads the application’s persisted state, selects the applicable configuration version, evaluates the event, and commits the resulting transition. Carrier callbacks and scheduled jobs enter through this same boundary.

External callbacks need authentication and correlation before becoming trusted workflow events. A carrier response must identify the correct tenant and application. The engine should never infer those identities from an arbitrary state-machine payload.

The frontend then consumes a server-generated view of current status and available actions. It does not independently reconstruct eligibility from employer settings. Backend endpoints still enforce authorization and transition rules because hiding a button does not prevent a direct API request.

This separation makes personalization a projection of workflow state. An employee awaiting documents sees an upload task because the application is in pending_documents, rather than because several frontend flags happen to align.

Reuse Workflow Fragments Without Hiding Dependencies

Copying a complete state machine for every employer simply relocates duplication into XML. A more maintainable structure uses shared lifecycle fragments and thin tenant-specific definitions.

An employer configuration selects a workflow variant and supplies values such as enrollment dates, supported regions, and reminder intervals. A standard application fragment handles ordinary review. A documentation variant adds the required intermediate state.

The source architecture proposes XInclude for assembling these fragments. Treat that assembly as an explicit preprocessing stage, then validate the resulting standalone SCXML document before activation.

Shared fragments also create shared risk. Changing one fragment can affect every employer that includes it. Version the resolved configuration and its dependencies together so reviewers can identify the exact tenants affected.

Prefer parameters when behavior differs only by a value. Introduce a workflow variant when the permitted sequence genuinely changes. Excessive variants make the configuration catalog as difficult to maintain as the conditionals it replaced.

Make Event Processing Safe to Retry

A correct transition graph does not guarantee correct distributed execution. Carrier callbacks can arrive twice, scheduled events can be delayed, and two workers can process the same application concurrently.

Persist an event identifier alongside the application identifier and enforce uniqueness. Use optimistic concurrency or equivalent serialization so two workers cannot successfully advance the same stored version.

Commit the state change, processed-event record, and any outbound message intent in one database transaction. A separate dispatcher can deliver notifications or integration requests from an outbox. Downstream handlers still need idempotency because delivery may repeat.

Define handling for unexpected events as well. A duplicate approval may be harmless, while an approval arriving after a denial may require investigation. The application must distinguish these cases instead of silently treating every unmatched event as success.

These responsibilities belong to the execution platform. Requiring each employer configuration to implement them would undermine reuse.

Treat Hot Reload as a Release

Runtime reload can avoid rebuilding the application when supported business parameters change. It still changes production behavior and deserves a controlled release process.

Consider an enrollment deadline extension. Updating the date does not automatically reevaluate applications that previously failed eligibility checks. A scheduler or targeted reevaluation job must emit an event that causes the relevant guard to run again.

Configuration changes also raise a compatibility question: which definition governs applications already in progress?

One approach applies the new definition at the next event while preserving current state. This works only when the new model still understands that state and its expected transitions. Removing pending_documents could strand applications already waiting there.

A safer default for structural changes is to pin existing applications to their configuration version. Migrate them through an explicit, tested procedure when necessary. Compatible parameter updates can follow a separately defined policy.

Load and validate a candidate configuration before atomically activating it. Retain the previous version, but remember that restoring XML does not reverse notifications or external requests already sent.

Validate Behavior, Not Just XML

Schema validation catches malformed definitions. It cannot establish that an enrollment workflow behaves correctly.

Add checks for missing targets, unreachable states, unsupported events, and required data. Then exercise representative event sequences, including deadline boundaries, repeated callbacks, documentation resubmission, and configuration changes during active applications.

Graph reachability is only a starting point. A transition can exist structurally while its guard prevents every realistic application from taking it.

Record the previous state, event, resulting state, configuration version, and correlation identifier for each decision. Monitor rejected events and unusually long state residence times. Those signals reveal workflows that are valid on paper but stalled in production.

The payoff is a clearer location for variability. New employers can reuse supported states, guards, and integrations without new application branches. New capabilities still require engineering, and configuration complexity still needs management. An explicit lifecycle makes both boundaries easier to see and change safely.