Deploy Application on Open-Shift that Requires Elevated Privileges on Specific Paths

For this document, we have created an Angular UI application and containerized it using NGINX for OpenShift 4.3.

Deploying Application by Updating Dockerfile

By default, OpenShift Container Platform runs containers using an arbitrarily assigned user ID. This provides additional security against processes escaping the container due to a container engine vulnerability and thereby achieving escalated permissions on the host node.

For an image to support running as an arbitrary user, directories and files that may be written to by processes in the image should be owned by the root group and be read/writable by that group. Files to be executed should also have group execute permissions.

Adding the following to your Dockerfile sets the directory and file permissions to allow users in the root group to access them in the built image:

RUN chgrp -R 0 /some/directory && \

    chmod -R g=u /some/directory

Note: /some/directory is the path on which image needs privileged access.

Because the container user is always a member of the root group, the container user can read and write these files. The root group does not have any special permissions (unlike the root user) so there are no security concerns with this arrangement. In addition, the processes running in the container must not listen on privileged ports (ports below 1024), since they are not running as a privileged user. 

Sample Dockerfile of our NGINX + Angular Application

x

FROM nginx:1.13.3-alpine

RUN rm -rf /etc/nginx/nginx.conf.default && rm -rf /etc/nginx/conf.d/default.conf

COPY /nginx/nginx.conf /etc/nginx/nginx.conf

COPY /nginx/nginx.conf /etc/nginx/conf.d/nginx.conf

## Remove default nginx index page

RUN rm -rf /usr/share/nginx/html/*

# Copy code from dist folder to nginx folder

COPY  /dist/my-angular-app/ /usr/share/nginx/html

RUN chgrp -R 0 /var/cache/ /var/log/ /var/run/ && \

    chmod -R g=u /var/cache/ /var/log/ /var/run/

EXPOSE 9090

#Entry point of application

ENTRYPOINT ["nginx", "-g", "daemon off;"]

In case the team wants specific user id during execution, the following steps can be followed:

Because the user ID of the container is generated dynamically, it will not have an associated entry in /etc/passwd. This can cause problems for applications that expect to be able to look up their user ID. One way to address this problem is to dynamically create a passwd file entry with the container’s user ID as part of the image’s start script. This is what a Dockerfile might include:

xxxxxxxxxx

RUN chmod g=u /etc/passwd

ENTRYPOINT [ "uid_entrypoint" ]

USER 1001

Where uid_entrypoint contains: 

xxxxxxxxxx

if ! whoami &> /dev/null; then

  if [ -w /etc/passwd ]; then

    echo "${USER_NAME:-default}:x:$(id -u):0:${USER_NAME:-default} user:${HOME}:/sbin/nologin" >> /etc/passwd

  fi

fi

Study more about using User Ids

Deploying Same Application by Using Service Accounts

Service Accounts Overview

A service account is an OpenShift Container Platform account that allows a component to directly access the API. Service accounts are API objects that exist within each project. Service accounts provide a flexible way to control API access without sharing a regular user’s credentials.

When you use the OpenShift Container Platform CLI or web console, your API token authenticates you to the API. You can associate a component with a service account so that they can access the API without using a regular user’s credentials. For example, service accounts can allow:

• Replication controllers to make API calls to create or delete pods.

• Applications inside containers to make API calls for discovery purposes.

• External applications to make API calls for monitoring or integration purposes.

Each service account’s user name is derived from its project and name:

xxxxxxxxxx

system:serviceaccount:<project>:<name>

Every service account is also a member of two groups:

• system:serviceaccounts Includes all service accounts in the system.

• system:serviceaccounts: Includes all service accounts in the specified project.

Each service account automatically contains two secrets:

• An API token

• Credentials for the OpenShift Container Registry

The generated API token and registry credentials do not expire, but you can revoke them by deleting the secret. When you delete the secret, a new one is automatically generated to take its place.

Default Project Level Service Accounts and Roles

Three service accounts are automatically created in each project:

All service accounts in a project are given the system:image-puller role, which allows pulling images from any imagestream in the project using the internal container image registry.

Note : Our strategy is to use default service account for pulling image via import-image(image stream) and creating deployment config. But for deploying our application we will our custom service account as we will give it sufficient access.

Before creating service account, we need to login to correct project inside our cluster :

x

[root@hou0posa01 ~]# oc project

Using project "test" on server "https://<hostname>:<port>".

Creating Service Account

1. To create a new service account in the current project:

xxxxxxxxxx

$ oc create sa <service_account_name> serviceaccount "robot" created

Exmaple: `oc create sa robot`

2. Optional: View the secrets for the service account

x

[root@hou0posa01 ~]# oc describe sa robot

Name:                robot

Namespace:           test

Labels:              <none>

Annotations:         <none>

Image pull secrets:  robot-dockercfg-q2x7n

Mountable secrets:   robot-dockercfg-q2x7n

                     robot-token-6n8pw

Tokens:              robot-token-6n8pw

                     robot-token-rdkpn

Events:              <none>

3. Before we give privileges to our service account, we need to understand SCC 

** Security Context Constraints (SCC)** It is basically used for pod restriction, which means it defines the limitations for a pod, as in what actions it can perform and what all things it can access in the cluster.

OpenShift provides a set of predefined SCC that can be used, modified, and extended by the administrator.

xxxxxxxxxx

[root@hou0posa01 ~]# oc get scc

NAME               AGE

anyuid             38d

hostaccess         38d

hostmount-anyuid   38d

hostnetwork        38d

node-exporter      38d

nonroot            38d

privileged         38d

restricted         38d

4. To give more privileges to our service account so that images can get privileged access, we will use anyuid SCC

• To add an SCC to a service account:

xxxxxxxxxx

$ oc adm policy add-scc-to-user <scc_name> \

    system:serviceaccount:<serviceaccount_namespace>:<serviceaccount_name>

Example from our openshift cluster:

xxxxxxxxxx

 oc adm policy add-scc-to-user anyuid system:serviceaccount:test:robot 

• If you are currently in the project to which the service account belongs, you can use the -z flag and just specify the <serviceaccount_name>.

xxxxxxxxxx

$ oc adm policy add-scc-to-user <scc_name> -z <serviceaccount_name>

Example from our cluster:

xxxxxxxxxx

oc adm policy add-scc-to-user anyuid -z robot

Usage of the -z flag as described above is highly recommended, as it helps prevent typos and ensures that access is granted only to the specified service account. If not in the project, use the -n option to indicate the project namespace it applies to.

5. We also need to create an Image pull secret as in our case image is present of Azure Container registry.

xxxxxxxxxx

$ oc create secret docker-registry <pull_secret_name> \

    --docker-server=<registry_server> \

    --docker-username=<user_name> \

    --docker-password=<password> \

    --docker-email=<email>

Note: we create image pull secret with name image-pull-test

6. Now we link pull secrets with default, builder & robot service accounts.

xxxxxxxxxx

oc secrets link default image-pull-test  --for=pull oc secrets link robot image-pull-test  --for=pull

Note: we have created our service account which we will use as deployer as it will grant privileges to pods.

7. Once we have the pull secret in place and is linked to pull images, we can use the azure image stream to fetch the image tag for the local OCP cluster image registry:

xxxxxxxxxx

# oc import-image sample-angular-app:latest --from=<registry-name>/sample-angular-app:latest --confirm

8. Once image stream is created, we need to create an application from it.

xxxxxxxxxx

oc new-app <image-stream name> --name <application name>

9. Before we expose route we need to update deployment config so, we first need to find deployment config of our application

x

[root@hou0posa01 ~]# oc get dc

NAME                            REVISION   DESIRED   CURRENT   TRIGGERED BY

sample-angular-app               5          1         1         config,image(sample-angular-app:latest)

Since we were working with sample-angular-app  our dc(deployment-config) is sample-angular-app .

so we now need to edit it

[root@hou0posa01 ~]# oc edit dc/sample-angular-app

It opens yml config in vi editor, we then added our custom service account in spec section:

x

 spec:

      containers:

      - image: <registry-name>/sample-angular-app@sha256:c03de1fc38e7878787889d5bb9eac0000008c478d14bb8fd59dd1cef9be809cea

        imagePullPolicy: Always

        name: sample-angular-app

        ports:

        - containerPort: 9090

          protocol: TCP

        resources: {}

        terminationMessagePath: /dev/termination-log

        terminationMessagePolicy: File

      dnsPolicy: ClusterFirst

      restartPolicy: Always

      schedulerName: default-scheduler

      securityContext: {}

      serviceAccount: robot

      serviceAccountName: robot

      terminationGracePeriodSeconds: 30

10. We then need to expose service i.e create a route. oc expose service <service name>

Reference link:

• https://docs.openshift.com/container-platform/4.1/authentication/using-service-accounts-in-applications.html

• https://blog.openshift.com/pushing-application-images-to-an-external-registry/

• https://docs.openshift.com/container-platform/3.6/dev_guide/service_accounts.html

• https://docs.openshift.com/container-platform/3.5/admin_guide/manage_scc.html#creating-new-security-context-constraints

• https://docs.openshift.com/container-platform/3.5/admin_guide/manage_scc.html

• https://www.tutorialspoint.com/openshift/openshift_security.htm

Running Pods With Particular UIDs – SCC Exploration With OpenShift

By default, OpenShift pods are brought up, and run as a random UID within a set range. This is a feature of a the default ‘restricted’ SCC in-place for the default service account. In OpenShift, pods are granted admission of privilege based on what service account a pod is started.

Let’s try to deploy a sample application that shows this in action.

Using specific user ids.

Conclusion

If the dev team already knows the directories on which privileged access is required and the number of directories on which privileges are required are not too much, then the first approach should be used. An example of the first approach would be our POC in which we deployed Angular UI application using NGINX.

The second approach is more useful if we require to install software on Openshift cluster and we don't know how many directories require privileges access. Also in case some directories are created dynamically during software execution, then the service account approach would be required. An example of the second approach would be to install a DB or any other software not present in OpenShift Catalog.