How to Setup MuleSoft Runtime Fabric on Self-Managed Kubernetes With AKS

Introduction

MuleSoft announced the general availability of Anypoint Runtime Fabric on Azure Kubernetes Service (AKS) and Amazon Elastic Kubernetes Service (EKS) with the v1.8 release. Release 1.8 allows two flavors of Runtime Fabric as follows:

1. Runtime Fabric with Mule Appliance.
2. Runtime Fabric with BYOK (Bring your Own Kubernetes).

RTF with Mule appliance is shipped with mule appliance and is packaged with its own Kubernetes. Therefore, it allows enterprises and implementing partners to take advantage of cutting-edge technologies such as Kubernetes and docker without knowing much about them. However, these days, most enterprises are moving towards the cloud and have embraced these technologies. 

Azure Kubernetes Service (AKS) and Amazon Elastic Kubernetes Service (EKS) are two dominant Kubernetes services adopted by most enterprises. These enterprises are happy to explore the idea of having the mule runtime managed on their own infrastructure with AKS or EKS if they see an advantage compared to RTF with mule appliances. This is what makes the proposition interesting. 

Runtime fabric with mule appliance needs extra infrastructure to run the controllers. This increases your infrastructure bill by some margin. However, if you go for Runtime Fabric with AKS or EKS, you can leverage your own Kubernetes service and expertise to manage the mule runtimes with a lot more flexibility, and overall infrastructure expenses will be reduced significantly. 

This article is your first step to get acquainted with Runtime Fabric setup on AKS.

Prerequisites

These are some of the prerequisites you will require before you begin with the Runtime Fabric Setup:

1. Active Azure Subscription. You can your free Azure account here.
2. Valid Anypoint Platform Account with RTF Entitlement. Talk to your MuleSoft PAM to enable RTF on your Anypoint Platform Org.
3. Azure CLI version 2.0.64 or later. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.

There are few more tools we may need, but we can get them while cruising through this article.

Step 1: Create the Kubernetes Cluster

1. Log in with your Azure Credentials.
2. Search for Kubernetes services and select Kubernetes services under the services section.
3. Click on Add -> Add Kubernetes Cluster from the top menu of the Kubernetes Services page.
   

   
   

   On the basic tab, fill in the following details:
   1. Select Valid Subscription.
   2. Select the existing Resource Group or provide the name of the new Resource Group.
   3. Provide the Kubernetes cluster name.
   4. Select the region in which you want to deploy your Kubernetes cluster.
   5. Select the availability zones for a given region. This will ensure the nodes of the cluster will spread across multiple physical locations to protect the application from azure datacentre failure. Learn more on availability zones for AKS here.
   6. Select the appropriate Kubernetes version. At the point when this article was written MuleSoft is supporting the Kubernetes version from 1.16.x to 1.18.x. Though we have tested RTF deployment with some higher versions, it is recommended to follow the guidelines provided by MuleSoft.
   7. Select the Node Size. Please note that once you select the size, all your nodes will be created with that size. Therefore, make sure you have done the sizing exercise before selecting the node size. The node size cannot be changed once the cluster is created.
   8. Select the number of nodes you require for your AKS cluster. You can resize the cluster by increasing nodes when needed.

      
      

   9. Click on Next : Node pools > Button
4. On Node Pool Tab, leave the default settings and click on the Authentication Button at the bottom of the page.

   
   

5. On the Authentication Tab: 
   1. Leave the Authentication method as *System-assigned managed identity*.

      Note: You can choose Service Principle if you have the existing principal’s SPN ClientID and secret.

   2. Role-based access control (RBAC) = Enabled. This will ensure fine-grained control over cluster Resources.
   3. AKS-managed Azure Active Directory = Disabled. (You may want to enable this setting if you have Azure Active Directory. Once this feature is enabled, it cannot be disabled)
   4. Leave the Node Pool encryption as default.
   5. Click on the Networking button at the bottom.

      
      

6. Accept default values on the rest of the pages and click the Review + Create button at the bottom of the page. Once Validation is passed, click the Create Button. After few minutes, your Kubernetes cluster will be ready.

   
   

Step 2: Connecting to the Kubernetes Cluster

1. Search for Kubernetes Service and click on the Kubernetes cluster name that you have created. On top of the page, you will see Connect button. Click on connect button, which will bring up the Kubernetes cluster connect commands on the right. 

   
   

2. Copy the first command and paste the same into your Azure CLI PowerShell window.
3. Copy the second command and paste the same into the Azure CLI PowerShell window.

   
   

4. To validate the connection with the cluster, you can use the following command: 
   PowerShell

    

   kubectl get nodes

   
   

   Note: If you don’t have kubectl installed, you can get the same from the following link.

Step 3: Create a Runtime Fabric

 To install Runtime Fabric on Self-Managed Kubernetes, first, create a Runtime Fabric using Runtime Manager. You will see a runtime fabric option in your runtime manager if your Anypoint Platform has the appropriate entitlement. 

1. Check the Runtime fabric entitlement: Login to Anypoint Org and visit the following URL search for "runtimeFabric". It is true that if you found this entry, then your org does have RTF entitlement.
2. Make sure you have * Manage Runtime Fabrics* permissions in your org. 
3. Go to the Runtime manager in Anypoint Platform and click on runtime Fabric.

   
   

4. Click the Create Runtime Fabric Button.
5. Enter the Name of the Runtime Fabric and select Azure Kubernetes Service from the options and click next.

   
   

6. Read and understand support responsibility declaration, and then click on Accept.
7. Make a note of the activation code; you will need this later. Your RTF Status should be *Ready for Activation*. Copy the activation code; we will use this later.

   
   

Step 4: Validate the Runtime Fabric

1. Download the rtfctl utility. For windows, you can use the following command from the command prompt: curl -L.
2. Validate Runtime Fabric into your Kubernetes service with the following command in the windows command prompt. rtfctl validate <<Activation code copied from runtime manager>>
3. If your Kubernetes configuration is not located in the “/.kube/config” directory, set the KUBECONFIG environment variable before running rtfctl.
4. The validate option  rtfctlverifies the following:
   1. The Kubernetes environment is running.
   2. All required components exist.
   3. All required services are available.

Step 5: Install the Runtime Fabric into the AKS Cluster

Once validation is successful, Install Runtime Fabric with the following command on the Command Prompt:  rtfctl install <activation_data>

<activation_data> is the activation data obtained after creating the Runtime Fabric using Runtime Manager. During installation, the rtfctl utility displays any errors encountered.

Step 6: Insert Mule License Key

Please go to this link to download your mule license key.

If you are working on a Windows machine, you will require a Linux bash shell terminal for the base64 encoding of the license key. You can download the bash terminal here.

1. Go to your Linux bash shell terminal and Run the following command to Base64 encode the license key:   base64 -w0 license.lic
2. Make sure you have licence.lic available in the current directory or in the Cygwin installation folder. Ex : C:\cygwin64\home\ranjeet_bhoyar 
3. Go to command prompt and apply the mule license key with the following command: rtfctl apply mule-license BASE64_ENCODED_LICENSE
4. Check the runtime fabric you create on your Anypoint platform, it should be in an active state. 

   
   

5. To Verify mule license key applied correctly, run the following command: rtfctl get mule-license

Step 7: Create the Ingress Controller in the AKS Cluster

We will use Helm 3 to install the NGINX ingress controller on our Kubernetes cluster. Make sure that you are using the latest release of Helm and have access to the ingress-nginx Helm repository.

To create Ingress Controller, follow these steps:

1. Go to your PowerShell command prompt and execute the following command to get the kubectl tool:
   PowerShell

    

   curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/windows/amd64/kubectl.exe

2. Make sure that you are connected to the AKS cluster from your PowerShell window. If not, then connect again by using az account set and az aks get-credentials commands as mentioned in Step 2 of this article.
3. Create a namespace for the Ingress controller with the following command: ./kubectl create namespace ingress-basic
4. Add the ingress-nginx repository with the following command: ./helm repo add ingress-nginx01 https://kubernetes.github.io/ingress-nginx 
5. Use Helm to deploy an NGINX ingress controller: 
   PowerShell

    

   ./helm install nginx-ingress01 ingress-nginx/ingress-nginx 
   --namespace ingress-basic --set controller.replicaCount=2 
   --set controller.nodeSelector."beta\.kubernetes\.io/os"=linux 
   --set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux 
   --set controller.admissionWebhooks.patch.nodeSelector."beta\.kubernetes\.io/os"=linux

6. On a successful response, review the Ingress controller on the AKS cluster. Click on <<AKS clustern name>> under Kubernetes Service, and look for Services and Ingress Option. Notice that Ingress is created with the specified namespace. The Ingress controller will have an internal IP and external IP assigned. Have a look at the following screenshot:

   
   

7. Alternatively, use the following command to validate the ingress controller:kubectl --namespace ingress-basic get services -o wide -w nginx-ingress01-ingress-nginx-controller

   
   

   
   

Step 8: Validate the Runtime Fabric and Deploy the Application

After completing the installation, your Runtime Fabric should be activated within your Anypoint organization. To validate your installation, go to Anypoint Runtime Manager and confirm that the status of the Runtime Fabric is Active.

Before deploying an application to your Runtime Fabric, do the following:

1. Associate RTF with at least one Anypoint environment
2. Add A Record into your Domain Provider and make sure you can nslookup to the subdomain

   
   

   
   

3. Configure Domains and Inbound Traffic: 
   • Navigate to Runtime Manager and select Runtime Fabric.
   • Select the name of your Runtime Fabric (RTF02) to open the management page.
   • Select Inbound Traffic, then click Add Endpoints.
   • Add the following endpoint. RTF02.yourdomainname.com(this depends on your domain name and A Record configuration).
   • Confirm your changes by clicking the Add button.
   • Click the Save and deploy button, and add the new endpoint.
4. Deploy the Application:
   • Navigate to Runtime Manager -> Applications and Click on Deploy Application
   • Provide the name of the application. Note that the name of the application must be lowercase. (in Cloudhub, we can use Capital case in application name)
   • Select Deployment target as Runtime Fabric name you just created.
   • Upload the application’s JAR file.
   • Select the desired runtime version.
   • Select the number of replicas
   • Option *Run in Runtime Cluster-Mode* will be enabled only when you have more than one node and more than one in your AKS cluster and more than 1 replica selected.
   • Option *Enforce deploying replicas across node* can be selected if your AKS cluster nodes are greater than or equal to your replicas.
   • Use Persistent Storage: The Persistent gateway needs to be configured with RTF to use this option.
   • Select Deployment Model as Rolling Update. This ensures availability by incrementally updating replicas. Requires one additional replica is worth of resources to succeed.
   • Select Reserved CPU, CPU Limit, and Memory as per your sizing requirements.
   • Click on the Deploy Application Button.

Step 9: Custom Ingress Controller Configurations

1. Navigate to the AKS Cluster -> Service and Ingress. You will see your application deployed and the status as OK. 
2. Notice the namespace of the application deployed. This namespace corresponds to the environment ID in Anypoint Platform. The Runtime Fabric agent generates a service and ingress resources under this namespace.
3. Many times, generated ingress resources may not work, therefore, you may need to modify generated resources with a custom ingress template. Create a custom ingress template create a text file IngressConfig01.txt with the following text.
4. Plain Text

    

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: ingress-config
     namespace: rtf
   data:
     ingress.annotations: |
       kubernetes.io/ingress.class: nginx
       nginx.ingress.kubernetes.io/ssl-redirect: "false"
       nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
       nginx.ingress.kubernetes.io/rewrite-target: /$2
       nginx.ingress.kubernetes.io/use-regex: "true"
     ingress.spec: |	
       rules.http.paths.path: /${APP_PATH}(/|$)(.*)
   

   Note: Do not edit anything above the Data section. Settings in the Data section can be changed in your AKS cluster configuration.

5. In your Windows command prompt, run the following command: rtfctl apply ingress-configmap <custom-template-file>

Running this command, rtfctl validates the template file you provided and performs the following:

• Creates the ConfigMap file in the rtf namespace. Any errors are written to stdout.
• Schedules a job to validate the ingress resource generated from the ConfigMap in the template file.
• Waits until the job completes to output the status and relevant logs. rtfctl displays the status to stdout.

Step 10: Test Out the Application

Test your application with a valid endpoint.

Important Tip: By default, AKS clusters are case-sensitive for your endpoints. In the above example, if you provide positiveGrains or positivegrains, it will not work with the default configuration. The endpoint expects an exact match to what you have added to your application.