Cloning Code In Containers
Take a look at several methods you can use to clone code into Docker containers.
Join the DZone community and get the full member experience.Join For Free
One common situation that I run into on different projects is how to tell my Docker container to clone or pull code automatically. In this article, we will see the various ways of getting the code to the Docker container.
There are multiple ways of getting code into the Docker container. In this post, we will see how we can clone a Github public and private repo using oauth tokens and with ssh keys.
1. Using COPY or ADD Command
Docker provides both COPY and ADD instructions for adding files/code to the Docker image. A simple Dockerfile will look something like this:
FROM ubuntu MAINTAINER "jagadish Manchala" COPY . /home/sampleTest WORKDIR /home/sampleTest
The source code needs to be cloned to the current directory where the Dockerfile exists. Once we build the Docker image, the code is copied to the /home/sampleTest directory and will be available once we run the container.
COPY both work similarly.
ADD has the extra capability of fetching remote URLs and extracting tarballs.
ADD techniques are most commonly used in production to build production-ready Docker images
docker run -d --name my-source-project -v "$PWD":/tmp gitdocker
This command starts a container attaching the current working directory mounted as a volume to the container on /tmp.
This is a commonly used feature when developing code. We can use a favorite editor to edit the code locally and then attach the directory as a volume to the running container and check the changes.
2. Using the Volume Feature
The second method is the most commonly used way of sharing local content with the container. Using volumes we can mount a local directory to the container at runtime. The running container can share the files with the host machine.
We need to checkout the latest source code to a directory and mount the directory as a volume to the running container. This can be done as,
docker run -d --name my-source-project -v "$PWD":/tmp gitdocker
This command start a container attaching the current working directory mounted as a volume to the container on /tmp. This is commonly used feature when developing code. We can use a favorite editor to edit the code locally and then attach the directory as a volume to the running container and check the changes
3. Using Git Clone Directly
The third method is to run the
git clone command directly. Docker provides the
RUN instruction to execute the commands directly inside the container while building the Docker image. We can use the
RUN instruction with the
git clone command in the Dockerfile as below,
FROM ubuntu MAINTAINER "jagadish Manchala" #Install git RUN apt-get update \ apt-get install -y git RUN mkdir /home/sampleTest \ cd /home/sampleTest \ git clone https://github.com/jagadish12/SampleTest.git #Set working directory WORKDIR /home/sampleTest
Though this creates a security risk we can pull the public Github repo directly into the Docker image while building. The code will be available when we start the container. Review DZone's guide to container security.
4. Clone Code Using Github OAuth Tokens
We can securely pull the code from Github using the Dockerfile
RUN instruction. We will be using a Github OAuth token to access our repositories in an automated fashion in the Dockerfile.
Create Github OAuth Token
Using personal access tokens allows us to skip providing the username and password while making the request. This is a recommended approach to automating the automation of code. This is also an easier way then working with SSH Keys. Review our guide to OAuth token validation.
In order to generate the OAuth token, go to your Github -> Settings -> Applications -> Generate New Token. Github will ask for the access levels for the token. Choose the Repo level access permissions and generate the token. Save the token. DZone has a full list for GitHub security best practices.
Using the Token
Using the token is very easy. All we have to do is to provide the Github URL along with the token generated. From the Dockerfile, we can use the
RUN instruction along with the Git clone:
RUN mkdir -p /home/app RUN git clone -b master https://7dd7c748e8c0f376920911952fffc7210e03b6db:email@example.com/jagadish12/SampleTest.git /myapp/
The syntax for the git clone is,
git clone -b https://:firstname.lastname@example.org/jagadish12/SampleTest.git /myapp/
All we have to do is to pass the token along with the Github URL. This way we can clone the private repo using secure way. DZone's previously covered how to perform Docker health checks.
5. Using SSH Keys
Docker from version 18 provides us a way to use ssh keys for cloning the code while building the image itself. There are a few things that need to be done before using this feature.
The first thing that needs to be done is enabling the Buildkit backend. Buildkit is an opt-in feature in Docker version 18 that can be enabled with an environment variable
DOCKER_BUILDKIT=1 before running the Docker build. Run the
export DOCKER_BUILDKIT=1 command to set the variable.
The second thing that we need to do to use the buildkit is to enable the support by adding an experimental syntax in Dockerfile. We need to define the syntax directive as the first line of the Dockerfile. This feature is not enabled in the stable version of the Dockerfile so we need to use the experimental release by adding the syntax line as the first line in the Dockerfile. e.g. docker/dockerfile:experimentalor docker/dockerfile/1.0.0-experimental.
Before running the build for the above Docker image, we need to make sure to configure our host machine with Github.
1. Run the
ssh-keygen command and create the id_rsa and id_rsa.pub files.
2. Go to Github -> Repository -> Deploy keys. Create a new key and paste the id_rsa.pub file contents in there and give it a name. Make sure you can clone the repo using the ssh way.
3. The dockerfile looks something like below:
jagadishm@[/Volumes/Work/build]: cat Dockerfile # syntax=docker/dockerfile:1.0.0-experimental FROM centos AS build # install git RUN yum install -y git RUN mkdir -m 700 /root/.ssh; \ touch -m 600 /root/.ssh/known_hosts; \ ssh-keyscan github.com > /root/.ssh/known_hosts RUN --mount=type=ssh,id=github git clone email@example.com:jagadish12/simple-java-maven-app.git
We are aware of the most instructions. The most important one is the:
RUN --mount=type=ssh,id=github git clone firstname.lastname@example.org:jagadish12/simple-java-maven-app.git
With the new build kit feature we can use mounts in the
RUN instruction of the Dockerfile. In the above
RUN instruction we are using the mount type as ssh. In the above case, I have defined an id “GitHub” which will be used during the build. The value to the Github id will the private key file from our local machine. We will be creating the build image by running the
In the command we are running the build by passing the --ssh and passing the location of the private key file to the Github id that we used in the Dockerfile. The ssh key that is mounted is exposed to the single command that defined the mount, not to the other parts of the build. The ssh key mounted will not be available even in the image metadata. If we run the “docker <Image> history” ,we will not be seeing any details regarding the key mounted.
docker build --ssh github=/Users/jagadishmanchala/.ssh/id_rsa -t buildtest.
When we run this,
jagadishm@[/Volumes/Work/build]: docker build --ssh github=/Users/jagadishmanchala/.ssh/id_rsa -t buildtest . [+] Building 7.1s (12/12) FINISHED => [internal] load build definition from Dockerfile 0.1s => transferring dockerfile: 617B 0.0s => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.0s => resolve image config for docker.io/docker/dockerfile:1.0.0-experimental 2.6s => CACHED docker-image://docker.io/docker/dockerfile:1.0.0-experimental@sha256:d2d402b6fa1dae752f8c688d72066a912d7042cc172 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 617B 0.0s => [internal] load metadata for docker.io/library/centos:latest 0.0s => [1/4] FROM docker.io/library/centos 0.0s => CACHED [2/4] RUN yum install -y git 0.0s => CACHED [3/4] RUN mkdir -m 700 /root/.ssh; touch -m 600 /root/.ssh/known_hosts; ssh-keyscan github.com > /root/.ssh/ 0.0s => [4/4] RUN --mount=type=ssh,id=github git clone email@example.com:jagadish12/simple-java-maven-app.git 3.5s => exporting to image 0.1s => => exporting layers 0.1s => => writing image sha256:7a7c7b6731eccde3e5aa9bc5b186174cb88f19d383524fb67a5cae11fed95888 0.0s => => naming to docker.io/library/buildtest 0.0s
Once the image is built, we can run the container and see if the repository is cloned.
jagadishm@[/Volumes/Work/build]: docker run -it buildtest /bin/bash [root@536ee37212ad /]# ls anaconda-post.log dev home lib64 mnt proc run simple-java-maven-app sys usr bin etc lib media opt root sbin srv tmp var [root@536ee37212ad /]# exit
We can see that the project simple-java-maven-app is cloned successfully.
Note: The buildkit feature also supports forwarding ssh connections. So for cloning the command with a private key, the request is then forwarded to the host machine where the key is already configured. This key should be available and need to be added using the “ssh-add” command. Once the key is available to the agent, the Docker build will be using the agent to clone the command. This way we don't need to pass the keys to the builder also.
Opinions expressed by DZone contributors are their own.