Feature Flags and Canary Releases in Microservices

Feature flags are commonly used constructs and have been there for a while. But in the last few years, things have evolved and feature flags are playing a major role in delivering continuous risk-free releases. In general, when a new feature is not fully developed and we still want to branch off a release from the mainstream, we can hide our new feature and toggle it off in production. Another use-case is when we want to release our feature to only a small percentage of users, we set the feature 'on' for a segment/geography and set it 'off' for the rest of the world. The capability to toggle a feature on and off without doing a source code change gives the developer an extra edge to experiment with conflicting features with live traffic. Let us deep dive into more details about feature flags and an example implementation in Springboot. 

Things To Consider When We Are Introducing a New Feature Flag

1. Establish a consistent naming convention across applications, to make the purpose of the feature flags easily understandable by other developers and product teams.
2. Where to maintain feature flags?
   1. In the application property file: Toggle features based on environment. Useful for experimenting in development while keeping features off in production.
   2. In configuration server or vault: Let's imagine you are tired after a late-night release, and your ops team calls you at 4 am, to inform you the new feature is creating red alerts everywhere in monitoring tools, here comes the Feature toggle to your rescue. First, turn the feature 'off' in the config server and restart compute pods alone,
   3. In database or cache: Reading configs or flag values from a database or an external cache system like Redis, you don't have to redeploy or restart your compute, as the values can be dynamically read from the source at regular intervals, pods get updated value without any restart. 

You can also explore open-source or third-party SDKs built for feature flags, a handful of them are already in the market. They also come with additional advantages that help in the lifecycle management of feature flags. 

3. Set the blast radius: Always start with smaller ramp-up percentages in case of the new feature(which means only a bunch of users can experience this new feature) and increase gradually once you gain confidence. Everything goes very well in lower environments, but things might behave differently in prod, so it is advisable to start a new feature live only to a segment of users, it could be your own dev team accounts in prod. Monitor and collect data points on things like performance change or user experience change with the introduction of new features. Take it back to your team and evaluate its worthiness. Do a thorough testing and keep on increasing the ramp-up percentage to 100 once you have all data points favorable.  

4. Keep the feature flag close to the business logic layer. 

5. Try to maintain metadata around feature flags for future reference, like who created it, when it is created, the purpose of creation, and the expiry timestamp.

How To Maintain Feature Flags

As time passes by, maintaining feature flags can be a nightmare if unused ones are not removed at the end of their lifecycle. To manage the clutter effectively, include tasks in your backlog that serve as reminders for post-rollout cleanup in both code and configuration repositories. Another way to handle the toil is, to create every feature flag with a TTL (time to live), this could be a rough estimate time for complete rollout and once the TTL of a flag expires, we could log it in elf and have an alert in monitoring tools to notify us.

Let us see an example of implementing a feature flag in a Java SpringBoot application:

Assume we have to introduce a new feature in an e-commerce website, where user gets credited $5 in the order summary for every order they placed with the delivery type Pickup instead of Home Delivery.

We decided to roll out the release with a 1% ramp-up. The strategy to select those 1% of customers out of all is not in the scope of this article. 

Now, let's look into Java code.

Add a new property in application.property file of the dev profile.

 app.checkout-summary.delivery-pickup.enabled = true

Write a contract CheckoutSummary.java

public interface CheckOutSummary{

double calculateOrderTotal(Cart cart);

}

Providing two different implementations for the above contract. Here, instead of modifying the existing impl class with if and else clauses to introduce a switch, we use a decorator pattern. It helps us to extend the functionality of existing objects without modifying their source code.

public class CheckOutSummaryImpl{

double calculateOrderTotal(Cart cart)

{

//standard logic

}

}

public class PickUpCheckOutSummaryImpl{

public final CheckOutSummaryImpl checkoutSummary;

Public PickUpCheckOutSummaryImpl(CheckOutSummary checkoutSummary)

{

this.checkoutSummary = checkoutSummary;

}

double calculateOrderTotal(Cart cart)

{

 var newOrderAmt =  checkoutSummary.calculateOrderTotal() - 5;

return newOrderAmt;

}

}

Moving on to Configurations. We need to conditionally inject the above impl beans.

@Configuration

public class SummaryBeans{

@Bean

@ConditionalOnProperty(value="app.checkout-summary.delivery-pickup.enabled", havingValue="false")

public CheckOutSummary checkoutSummary()

{

return  new CheckOutSummaryImpl();

}



@Bean

@ConditionalOnProperty(value="app.checkout-summary.delivery-pickup.enabled", havingValue="true")

public CheckOutSummary checkoutSummaryWithPickUp( CheckOutSummary summary)

{

return  new PickUpCheckOutSummaryImpl(summary);

}

}

In addition to the above example, we can also create No Operation Implementations beans if no actions are required in case the flag is toggled off.

Sometimes, our feature flag may need to evaluate more than one condition to satisfy 'on' route. We can embed such conditions using the below spring annotation.

@ConditionalOnExpression("${properties.first.property.enable:true} && ${properties.second.property.enable:false}")

Instead of plain if and else blocks, writing implementations to contracts and conditionally creating beans looks more elegant and mature. 

In summary, feature flags are a great tool when used with caution. Eliminate the toil and maintain clean code by removing unused feature flags.