Contact Center Feature Flags Using AWS AppConfig

This article explains the utilization of AWS AppConfig for managing Amazon Connect cloud contact center IVR experiences/features, consideration, benefits, and technical implementation. This technical article intends to provide a solution recommendation to release a new contact center feature in production safely in a controlled manner. 

Releasing New Feature

To release a new IVR feature in production, the feature management solution must support the following requirements.

• Support static and dynamic business rules to enable a feature
• Ability to enable/disable features without deployments/downtime
• Ability to manage the features on the client and server-side
• Disable the feature automatically in case of issues
• Enable/Disable the feature across multiple product teams
• Support progressive rollouts and custom deployment strategies

The criteria of a good solution include: 

• Leverage feature flags to turn a feature ON/OFF
• Support for schema validation so that incorrect configuration is detected before the deployment
• Expose the feature flag configuration over HTTP(REST) for clients to access
• Clients should be able to cache the configuration and refresh it periodically so that overall performance
• Turn OFF the feature flag automatically in case of any issue specific to the released feature noticed during the change window
• Turn OFF the feature flag automatically in case of any issue specific to the released feature noticed outside of the change window
• APIs to deploy or roll back the configuration changes
• History of feature flag changes.
• Easy integration with CI/CD pipeline and Git

To address the above requirements and solution criteria, leverage AWS AppConfig which supports the following key capabilities:

• Feature Flags and multiple sources for configuration like AppConfig hosted configuration, S3, AWS Systems Manager Parameter, and AWS Secrets Manager
• Syntax validation through schema validation or Lambda function
• Progressive rollouts and rollback alarms
• Custom deployment strategies
• Change log
• AWS App Config Extension
• AWS Power Tools to define dynamic feature flags

Process Steps

The process of releasing a new feature using AWS AppConfig involves a phased approach to ensure controlled deployment. 

1. Setup a Feature Flag in AWS App Config (Version 1)

1. Begin by creating the feature flag with an initial status set to 'inactive.' This initial version is referred to as Version 1.
2. Deploy Version 1 of the feature flag along with all the code components required for the new feature.
3. At this point, the new changes are not active as the feature flag status is set to 'inactive'.

2. Deploy Application Code Components (With Version 1 Feature Flag)

1. All the necessary code components for the new feature are deployed alongside Version 1 of the feature flag.
2. Code references the feature flag for conditional logic.
3. Despite the deployment, the feature remains inactive due to the 'inactive' status of the feature flag.

3. Activation of the New Feature (Version 2)

1. To release the new feature, the feature flag needs to be made active.
2. Create another version of the same feature flag, labeling it as Version 2, and set its status to 'active'.
3. Deploy Version 2 of the feature flag with the 'active' status.

4. Activation of the New Feature

1. With the deployment of Version 2 of the feature flag, the new feature becomes active.
2. Users can now experience and interact with the newly released feature as the feature flag's status transitions from 'inactive' to 'active'.

By following this sequential process, developers can ensure a controlled and gradual release of new features, providing the flexibility to activate or deactivate features based on the desired timeline and conditions. This approach allows for a smooth transition from inactive to active states, minimizing the risk of potential issues and ensuring a seamless user experience while introducing new functionalities.

The following diagram illustrates the above steps for rolling out a new feature in Amazon Connect IVR using AWS App Config.

The following diagram explains the Amazon Connect IVR flow before rolling out a new feature. In this experience, the customer calls the IVR which provides a self-servicing experience for feature 1, feature 2, and feature 3. Example features include checking balance, making a payment ordering a new product, etc. 

Automatically Rollback the Config Changes

In such a scenario, the feature flag version is reverted to the previous iteration, for instance, transitioning from version 2 back to version 1, as illustrated in the aforementioned example. It is crucial to emphasize that this action specifically reverts the feature flag version and does not roll back all the previously deployed code components. The rollback process is confined to the feature flag configuration, ensuring a targeted and controlled adjustment without affecting the entire codebase.

During the deployment window, the AWS App Config monitors for pre-configured cloud watch alarms. In the case of the cloudwatch event, AWS config rollbacks the configuration value back to the previous version which would disable the feature.  

FAQs

What Are the Benefits of AWS AppConfig Over the Custom Database Solutions for Managing Feature Flags?

1. Database design just supports a simple feature flag as the schema is fixed.
2. Database change is needed for defining a slightly complex feature flag.
3. With AppConfig, it's a freeform text and the feature flag can be anything.
4. Supports validators so that it is error-free.
5. Supports a dynamic feature flag which would give different decisions based on the request parameters.
6. AWS provides Python 'power tools' to achieve this on the client side. 
7. Progressive rollouts, automatic rollbacks... etc. 

Will Appconfig Automatic Rollbacks Automatically Roll Back All the Code Changes Back to the Previous Working Version?

1. No, AppConfig just rolled back the feature flag configurations back to the previous state.  

How to Toggle the Features On/off Using AppConfig

1. Even though the features can be managed through the AWS console, it is recommended to use the CICD pipeline to manage the feature management.
2. Custom UI can be developed using AppConfig and Evidently REST services.

What Happens if AppConfig Goes Down?

1. It is not required to invoke AppConfig for every request as the feature flags will change often.
2. AWS supports pre-built extensions through which client applications can cache the data and reuse it. In case of any downtime, requests will be served through the data present in the cache.

How Is AWS AppConfig Different From Azure App Configuration?

Conclusion

By embracing the capabilities offered by AWS AppConfig, developers can navigate the complexities of controlled feature releases and efficiently manage the feature flags. In addition to AWS App Config, Cloudwatch Evidently can be used for multi-variant testing of a feature and get real-time insights.