NGINX Ingress Controller Configuration In AKS
Take a look at some of the different configurations NGINX's controller configuration.
Join the DZone community and get the full member experience.
Join For FreeIngress Controller
Ingress controller is a necessary Kubernetes feature that plays a vital role in the functioning of Ingress resources. The Ingress resources deployed in the cluster are controlled by the Ingress controller. As mentioned in Kubernetes Ingress controller page: "Unlike other types of controllers which run as part of the kube-controller-manager binary, Ingress controllers are not started automatically when a cluster is created." It must be deployed into the cluster manually and configured according to the prescribed requirements.
NGINX Ingress Controller for Kubernetes
According to the NGINX Ingress Controller page, "The NGINX Ingress Controller for Kubernetes provides enterprise-grade delivery of services for Kubernetes applications, with benefits for users of both open source NGINX and Kubernetes." We are using this NGINX Ingress Controller.
Deploying a NGINX Ingress Controller
kubectl apply -f
apiVersion v1
kind Namespace
metadata
name ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
---
kind ConfigMap
apiVersion v1
metadata
name nginx-configuration
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
---
kind ConfigMap
apiVersion v1
metadata
name tcp-services
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
---
kind ConfigMap
apiVersion v1
metadata
name udp-services
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
---
apiVersion v1
kind ServiceAccount
metadata
name nginx-ingress-serviceaccount
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
---
apiVersion rbac.authorization.k8s.io/v1beta1
kind ClusterRole
metadata
name nginx-ingress-clusterrole
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
rules
apiGroups
""
resources
configmaps
endpoints
nodes
pods
secrets
verbs
list
watch
apiGroups
""
resources
nodes
verbs
get
apiGroups
""
resources
services
verbs
get
list
watch
apiGroups
"extensions"
resources
ingresses
verbs
get
list
watch
apiGroups
""
resources
events
verbs
create
patch
apiGroups
"extensions"
resources
ingresses/status
verbs
update
---
apiVersion rbac.authorization.k8s.io/v1beta1
kind Role
metadata
name nginx-ingress-role
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
rules
apiGroups
""
resources
configmaps
pods
secrets
namespaces
verbs
get
apiGroups
""
resources
configmaps
resourceNames
# Defaults to "<election-id>-<ingress-class>"
# Here: "<ingress-controller-leader>-<nginx>"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
"ingress-controller-leader-nginx"
verbs
get
update
apiGroups
""
resources
configmaps
verbs
create
apiGroups
""
resources
endpoints
verbs
get
---
apiVersion rbac.authorization.k8s.io/v1beta1
kind RoleBinding
metadata
name nginx-ingress-role-nisa-binding
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
roleRef
apiGroup rbac.authorization.k8s.io
kind Role
name nginx-ingress-role
subjects
kind ServiceAccount
name nginx-ingress-serviceaccount
namespace ingress-nginx
---
apiVersion rbac.authorization.k8s.io/v1beta1
kind ClusterRoleBinding
metadata
name nginx-ingress-clusterrole-nisa-binding
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
roleRef
apiGroup rbac.authorization.k8s.io
kind ClusterRole
name nginx-ingress-clusterrole
subjects
kind ServiceAccount
name nginx-ingress-serviceaccount
namespace ingress-nginx
---
apiVersion apps/v1
kind Deployment
metadata
name nginx-ingress-controller
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
spec
replicas1
selector
matchLabels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
template
metadata
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
annotations
prometheus.io/port"10254"
prometheus.io/scrape"true"
spec
serviceAccountName nginx-ingress-serviceaccount
containers
name nginx-ingress-controller
image quay.io/kubernetes-ingress-controller/nginx-ingress-controller0.24.1
args
/nginx-ingress-controller
--configmap=$(POD_NAMESPACE)/nginx-configuration
--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
--udp-services-configmap=$(POD_NAMESPACE)/udp-services
--publish-service=$(POD_NAMESPACE)/ingress-nginx
--annotations-prefix=nginx.ingress.kubernetes.io
securityContext
allowPrivilegeEscalationtrue
capabilities
drop
ALL
add
NET_BIND_SERVICE
# www-data -> 33
runAsUser33
env
name POD_NAME
valueFrom
fieldRef
fieldPath metadata.name
name POD_NAMESPACE
valueFrom
fieldRef
fieldPath metadata.namespace
ports
name http
containerPort80
name https
containerPort443
livenessProbe
failureThreshold3
httpGet
path /healthz
port10254
scheme HTTP
initialDelaySeconds10
periodSeconds10
successThreshold1
timeoutSeconds10
readinessProbe
failureThreshold3
httpGet
path /healthz
port10254
scheme HTTP
periodSeconds10
successThreshold1
timeoutSeconds10
---
This will deploy the Ingress Controller deployment and other associated resources in the ingress-nginx namespace.
kubectl apply -f
:
kind Service
apiVersion v1
metadata
name ingress-nginx
namespace ingress-nginx
labels
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
spec
externalTrafficPolicy Local
type LoadBalancer
selector
app.kubernetes.io/name ingress-nginx
app.kubernetes.io/part-of ingress-nginx
ports
name http
port80
targetPort http
name https
port443
targetPort https
---
This will deploy the Ingress Controller LoadBalancer type Kubernetes service in the ingress-nginx namespace.
Ingresses
Ingress is a Kubernetes object that allows access to your Kubernetes services from outside the Kubernetes cluster. You can configure access by creating a set of rules that defines which inbound internet traffic must reach which Kubernetes service in the cluster.
For Ingress resources to work within a cluster you must deploy an Ingress Controller prior to deploying an Ingress. The Ingress controller allows the Internet traffic to enter in the Kubernetes cluster. Once it is set up correctly we can create Ingress resources in the cluster and route internet traffic to the services. Note that Ingress resources are namespace-specific.
Below is a sample code for an Ingress resource:
apiVersion extensions/v1beta1
kind Ingress
metadata
name service1-ingress
annotations
nginx.ingress.kubernetes.io/rewrite-target /
kubernetes.io/ingress.class nginx
nginx.ingress.kubernetes.io/enable-cors"true"
nginx.ingress.kubernetes.io/cors-allow-origin"*"
nginx.ingress.kubernetes.io/cors-allow-credentials"true"
nginx.ingress.kubernetes.io/cors-allow-methods"*"
spec
rules
host aks.example.com
http
paths
backend
serviceName service1
servicePort8089
Common Issues Faced when Using Nginx Ingress Controller with Default Settings
Headers and Body play vital roles in performing operations to an API service running in AKS. Ingresses allow the services running in AKS to talk to the outside world seamlessly. However one might face blockers when appropriate annotations and custom headers are not allowed in the configurations of the Nginx Ingress controller while the traffic hits the API service. There are a couple of ways to manipulate the configurations of the Ingress controller. One is by adding config maps that are connected to the Ingress controller deployment which affects the entire cluster, and the other is by adding necessary annotations to the Ingress resource, this way of manipulating configurations can affect only the namespace in which the Ingress resource is deployed.
Following are some of the common issues which we faced.
Rewrite Target
In the Ingress resource one must add nginx.ingress.kubernetes.io/rewrite-target annotations to redirect the internet traffic to a specific service of the application deployed in the cluster. The path
key in the below YAML snippet has the value with which the URL will get rewritten. For instance, if the URL is http://aks.example.com/service1, then the URL will get rewritten as http://aks.example.com and then accessed inside the pod.
apiVersion extensions/v1beta1
kind Ingress
metadata
name service1-ingress
annotations
nginx.ingress.kubernetes.io/rewrite-target /
spec
rules
host aks.example.com
http
paths
path /service1
backend
serviceName service1
servicePort80
Blocked Custom-Headers
Ingresses, by default block the custom header which are used by a API service running in the AKS. These custom headers must be allowed through ingress controller so that it reaches the API service from the outside world. To fix this issue we make use of annotations and config maps that adhere to the Nginx ingress controller.
We create this config map following is YAML code for the config map.
apiVersion v1
data
X-Different-Name"true"
X-Request-Start t=$ msec
X-Using-Nginx-Controller"true"
kind ConfigMap
metadata
name custom-headers
namespace ingress-nginx
We also need to add the annotation to the Ingress of that service as shown below.
apiVersion extensions/v1beta1
kind Ingress
metadata
annotations
kubernetes.io/ingress.class nginx
nginx.ingress.kubernetes.io/cors-allow-credentials"true"
nginx.ingress.kubernetes.io/cors-allow-headers Ocp-Apim-Subscription-Key,Ocp-Apim-Trace, Authorization, Content-Type
Ocp-Apim-Subscription-Key and Ocp-Apim-Trace are custom headers which we need to use to make use of Azure API management resource.
Underscores in Headers Are Not Allowed
Application developers have a habit of creating API services which accept headers that may have underscores in them, by default, the Nginx ingress controllers do not accept custom headers and headers containing underscores. To fix this issue, we edit the config maps that adheres to Nginx ingress controller.
When you create an Ingress controller it also creates a default config map know as nginx-configuration we edit this config map and add data to it. Following is YAML code for the config map.
apiVersion v1
data
enable-underscores-in-headers"true"
proxy-set-headers ingress-nginx/custom-headers
kind ConfigMap
metadata
labels
app ingress-nginx
name nginx-configuration
namespace ingress-nginx
Limited Proxy Body Size
In some cases, the body of an API service requires data as a payload which is of larger size than the limit. To increase the size of data supplied as the payload of the API we need to increase the PROXY BODY SIZE
allowed by the Ingresses.
To fix this issue we need to use the annotations in the ingresses. We use nginx.ingress.kubernetes.io/proxy-body-size: 10m. annotation in the Ingress of that particular service.
apiVersion extensions/v1beta1
kind Ingress
metadata
annotations
nginx.ingress.kubernetes.io/proxy-body-size 10m
Enable Cross-Origin Resource Sharing (CORS) in Ingress
According to Wikipedia, "Cross-origin resource sharing (CORS) is a mechanism that allows restricted resources on a web page to be requested from another domain outside the domain from which the first resource was served".
To enable CORS we need to add the three main annotations in the ingresses as shown below.
apiVersion extensions/v1beta1
kind Ingress
metadata
annotations
nginx.ingress.kubernetes.io/cors-allow-methods'*'
nginx.ingress.kubernetes.io/cors-allow-origin'*'
nginx.ingress.kubernetes.io/enable-cors"true"
Similarly, based on application needs there are many annotations which you can use to enable the features in the Ingress.
For more information on annotation of Ingress follow the link.
Opinions expressed by DZone contributors are their own.
Comments