How Do You Integrate Emissary Ingress With OPA?

API gateways play a vital role while exposing microservices. They are an additional hop in the network that the incoming request must go through in order to communicate with the services. An API gateway does routing, composition, protocol translation, and user policy enforcement after it receives a request from the client and then reverse proxies it to the appropriate underlying API. As the API gateways are capable of doing the above-mentioned tasks, they can be also configured to send the incoming client requests to an external third-party authorization (AuthZ) server. The fate of the incoming request then depends upon the response from this external auth server to the gateway. This is exactly where Open Policy Agent (OPA) comes into the picture.

There are many open-source Kubernetes native API gateways out there like Contour, Kong Gateway, Traefik, Gloo, etc. In this article, we will be exploring the Emissary Ingress.

Let’s dive deep and start understanding more bit about Emissary Ingress.

What Is Emissary Ingress?

Emissary Ingress was earlier known as Ambassador API gateway; it is an open-source Kubernetes native API gateway and is currently a CNCF Incubation Project. Like many other Kubernetes gateways, Emissary has also been built to work with Envoy Proxy. It is deployed as complete stateless architecture and supports multiple plugins such as traditional SSO authentication protocols (e.g., OAuth, OpenID Connect), rate limiting, logging, and tracking service. Emissary utilizes its ExtAuth protocol in the AuthService resource to configure the authentication and authorization for incoming requests. ExtAuth supports two protocols: gRPC and plain HTTP. For the gRPC interface, the external service must implement Envoy’s external_auth.proto.

OPA

Open Policy Agent is a well-known general-purpose policy engine and has emerged as a policy enforcer across the stacks, be it API gateways, service meshes, Kubernetes, microservice, CICD, or IAC. OPA decouples decision-making from policy enforcement such that whenever your software needs to make a decision regarding the incoming requests, it queries OPA. OPA-Envoy extends OPA with a gRPC server that implements the Envoy External Authorization API, thus making itself compatible to be an external AuthZ server to Emissary.

Integrating Emissary Ingress With OPA

The above figure shows the high-level architecture of Emissary and OPA integration. When an incoming request from a client reaches Emissary, it sends an authorization request to OPA, which contains input JSON. OPA evaluates this JSON against the Rego policies provided to it and responds to Emissary; if this result JSON from OPA has allow as true, then only the client request is further routed to API, or else the request is denied by Emissary and never reaches the API. We will be installing Emissary Ingress and integrating it with OPA for external authorization.

Getting Started

First, we will be needing to start a Minikube cluster. If you don’t have Minikube, you can install it from here.

minikube start

Install the Emissary Ingress to the minikube through Helm.

# Add the Repo:
helm repo add datawire https://app.getambassador.io
helm repo update
 
# Create Namespace and Install:
kubectl create namespace emissary && \
kubectl apply -f https://app.getambassador.io/yaml/emissary/2.2.2/emissary-crds.yaml
kubectl wait --timeout=90s --for=condition=available deployment emissary-apiext -n emissary-system
helm install emissary-ingress --namespace emissary datawire/emissary-ingress && \
kubectl -n emissary wait --for condition=available --timeout=90s deploy -lapp.kubernetes.io/instance=emissary-ingress

Or go to Emissary Ingress Documentation to install it through Kubernetes YAMLs.

Configuring the Routing for Demo Application

Different gateways have their own set of configurations for exposing a service. In Emissary, we need to configure the routing through Mappings and Listeners.

Mapping resource simply tells Emissary which service to redirect the incoming request to. It is highly configurable like Ingress. We will create a simple Mapping resource that will redirect all the incoming requests to our demo application’s service, which is demo-svc.

cat <<EOF | kubectl apply -f -
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
  name: demo-app-mapping  
spec:
  hostname: "*"
  prefix: /
  service: demo-svc
EOF

The Listener resource instructs Emissary to listen on the network for the incoming request. Here we will create a listener to listen on port 8080 and HTTP protocol and associate with hosts in All namespace. 

cat <<EOF | kubectl apply -f -
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
  name: demo-app-listener-8080
  namespace: emissary
spec:
  port: 8080
  protocol: HTTP
  securityModel: XFP
  hostBinding:
    namespace:
      from: ALL
EOF

Install the Demo Application

Install a simple echo server as a demo application.

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-app
  template:
    metadata:
      labels:
        app: demo-app
    spec:
      containers:
      - name: http-svc
        image: gcr.io/google_containers/echoserver:1.8
        ports:
        - containerPort: 8080
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
---
apiVersion: v1
kind: Service
metadata:
  name: demo-svc
  labels:
    app: demo-app
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: demo-app
EOF

Communicate with the demo app at different paths.

minikube service emissary-ingress -n emissary

Note: The above exposing method may not work for macOS users. They can use a busy box and configure it to hit the emissary local endpoint instead.

Copy the private URL with target port 80. The URL must be IP 192.168.49.2 followed by a NodePort like http://192.168.49.2:30329. Export the NodePort value to $NODEPORT environment variable and curl to that at paths as follows:

curl http://192.168.49.2:$NODEPORT/public

and

curl http://192.168.49.2:$NODEPORT/secured

OPA has not yet been added to the setup, and the above curl requests are directly sent to API without any policy enforcement.

How To Install and Configure OPA

OPA will be reading the policies fed to it via a ConfigMap. Create the following ConfigMap, which contains a policy that allows all incoming requests only through GET method.

cat <<EOF | kubectl apply -n emissary -f  -
apiVersion: v1
kind: ConfigMap
metadata:
  name: demo-policy
data: 
  policy.rego: |-
    package envoy.authz

    default allow = false

    allow {
       input.attributes.request.http.method == "GET" 
    }
EOF

OPA can be configured as an external authorization server via deploying it as an independent deployment or as a sidecar to the emissary-ingress. Here we will add it as a sidecar. Save the following YAML as opa-patch.yaml.

spec:
  template:
    spec:
      containers:
      - name: opa
        image: openpolicyagent/opa:latest-envoy
        ports:
        - containerPort: 9191
        args:
          - "run"
          - "--server"
          - "--addr=0.0.0.0:8181"
          - "--set=plugins.envoy_ext_authz_grpc.addr=0.0.0.0:9191"
          - "--set=plugins.envoy_ext_authz_grpc.query=data.envoy.authz.allow"
          - "--set=decision_logs.console=true"
          - "--ignore=.*"
          - "/policy/policy.rego"
        volumeMounts:
          - mountPath: /policy
            name: demo-policy
            readOnly: true
      volumes:
      - name: demo-policy
        configMap:
          name: demo-policy

Patch the emissary-ingress deployment and wait for the all the emissary-ingress pods to restart.

kubectl patch deployment emissary-ingress -n emissary --patch-file opa-patch.yaml

Wait until all the emissary-ingress pods come to Running state with OPA sidecar.

Create the following AuthService. AuthService is a resource that configures Emissary to communicate with an external service for Authn and AuthZ of incoming requests. We are configuring it to communicate with OPA on localhost since OPA is deployed as a sidecar.

cat <<EOF | kubectl apply -f -
apiVersion: getambassador.io/v3alpha1
kind: AuthService
metadata:
  name: opa-ext-authservice
  namespace: emissary
  labels:
    product: aes
    app: opa-ext-auth
spec:
  proto: grpc
  auth_service: localhost:9191
  timeout_ms: 5000
  tls: "false"
  allow_request_body: true
  protocol_version: v2
  include_body:
    max_bytes: 8192
    allow_partial: true
  status_on_error:
    code: 504
  failure_mode_allow: false
EOF

Try doing curl now. Since the policy accepts requests coming through GET method and there are no restrictions on the path, both the request will get a 200 OK response.

curl -i http://192.168.49.2:$NODEPORT/public
curl -i http://192.168.49.2:$NODEPORT/private

Now let's edit the policy to accept incoming requests at path /public only, and requests to any other path will be denied.

cat <<EOF | kubectl apply -n emissary -f  -
apiVersion: v1
kind: ConfigMap
metadata:
  name: demo-policy
data: 
  policy.rego: |-
    package envoy.authz

    default allow = false

    allow {
       input.attributes.request.http.method == "GET"
       input.attributes.request.http.path == "/public" 
    }
EOF

Now restart the emissary ingress deployment for policy changes to take effect.

kubectl rollout restart deployment emissary-ingress -n emissary

Wait until all the emissary-ingress pods come to Running state after a restart.

Now do a curl request at path /public. It will be accepted, but at path /private, it will be denied by OPA with a 403 response, and hence the request will not reach the demo API.

curl -i http://192.168.49.2:$NODEPORT/public
curl -i http://192.168.49.2:$NODEPORT/private

The decision-making about the incoming request from the client to the exposed API can be decoupled to OPA as an external authorization server in the Emissary Ingress setup. OPA can be added as a plug-and-play policy enforcer to Emissary and any other gateways supporting the Envoy External Authorization API. We hope you found this post informative and engaging.