DevSecOps: Enhancing Security With Vulnerability Scanning of Images and Source Code in CI/CD

This guide shows how to use Syft, Grype, and Trivy to scan container images for vulnerabilities in Gitlab pipelines during release.

Aug. 03, 23 · Tutorial

Likes (2)

Comment

Save

4.4K Views

Many companies strive to adopt the DevOps approach for software development and delivery. Alongside this, they face increasing security challenges, leading to the implementation of new innovative software development methods.

The need for security in the software deployment process is evident. Therefore, integrating security into CI/CD workflows should be done carefully to account for the ever-evolving technological landscape.

DevSecOps is an important concept that provides an automated approach to integrating security into the software delivery lifecycle. In the context of container solutions, there are specific challenges in adding security controls.

Furthermore, when using open-source containers, many of them may contain known and unknown vulnerabilities. For many organizations, it is difficult to determine the security of their containers confidently. Hence, these tools provide additional capabilities and features that facilitate faster implementation. However, not all of them always align with the organization's security goals. The reality is that lacking the skill to design secure deployment pipelines can come at a high cost to a company.

This article presents a guide on setting up, running, and using Syft, Grype, and Trivy in CI/CD. It describes how to automatically configure the release process to scan container images for vulnerabilities with Gitlab pipelines.

Solution Overview

Syft, Grype, and Trivy are popular vulnerability scanning tools used in the software development and deployment process. Here's a brief overview of each tool:

Syft: Developed by Anchore, Syft is an open-source command-line tool that focuses on scanning container images for vulnerabilities. It provides a detailed analysis of the container's software bill of materials (SBOM) by inspecting package managers, libraries, and dependencies. Syft also has integrations with various container registries and orchestration platforms to simplify the scanning process.
Grype: Also developed by Anchore, Grype is another open-source vulnerability scanner specifically designed for container images. It scans the image's software composition analysis (SCA) data to identify any known vulnerabilities. Grype's strength lies in its fast scanning capability and ability to handle complex image layers and formats.
Trivy: Trivy is an open-source vulnerability scanner that specializes in container images, as well as operating systems and applications. It uses vulnerability databases from various sources, including NVD, Red Hat, and Ubuntu, to detect known vulnerabilities in container images. Trivy is easy to use, offers extensive configuration options, and provides rapid and reliable scanning results.

Get Started

For scanning, you can use an Alpine Linux image; let's assume the latest version. We embed an additional step before loading the collected image into the registry, saving dependency lists in the artifacts of the task, and storing the database in the cache.

Here is a complete set of scans and reports received:

    GitHub Flavored Markdown
   
 

   variables:
  GO_VERSION: '1.20'

stages:
  - build
  - scan
  - upload

build:
  stage: build
  tags:
    - docker
    - dind
  image: golang:1.20-bullseye
  variables:
    DOCKER_DRIVER: overlay2
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_VERIFY: 1
    DOCKER_TLS_CERTDIR: "/certs"
    DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
  before_script:
    - |
      TIME=30; PASS=0
      echo "Waiting for docker daemon..."
      until [ -f "${DOCKER_CERT_PATH}/ca.pem" ] || [ ${TIME} -lt ${PASS} ]
      do
        echo -n ".";PASS=`expr $PASS + 1`; sleep 1
      done
  script:
    - make gobuild
  artifacts:
    name: "$CONTAINER_NAME"
    expire_in: 1h
    paths:
      - build/

upload:
  stage: upload
  needs:
    - scan
  dependencies: ["build"]
  tags:
    - docker
    - dind
  image: docker:23.0.3
  variables:
    DOCKER_DRIVER: overlay2
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_VERIFY: 1
    DOCKER_TLS_CERTDIR: "/certs"
    DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
  before_script:
    - |
      TIME=30; PASS=0
      echo "Waiting for docker daemon..."
      until [ -f "${DOCKER_CERT_PATH}/ca.pem" ] || [ ${TIME} -lt ${PASS} ]
      do
        echo -n ".";PASS=`expr $PASS + 1`; sleep 1
      done
  script:
    - |
      echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER $CI_REGISTRY --password-stdin
      docker build -t $CI_COMMIT_REF_SLUG:$CI_COMMIT_REF_SLUG .
      docker push $CI_COMMIT_REF_SLUG:$CI_COMMIT_REF_SLUG
      docker save $CI_COMMIT_REF_SLUG:$CI_COMMIT_REF_SLUG -o build/$CI_PROJECT_NAME.tar
  artifacts:
    name: "$CONTAINER_NAME"
    expire_in: 1h
    paths:
      - build/

scan_cve:
  stage: scan
  image: alpine:latest
  dependencies: ["build"]
  variables:
    GIT_STRATEGY: clone
    SYFT_REGISTRY_AUTH_AUTHORITY: "$CI_REGISTRY"
    SYFT_REGISTRY_AUTH_USERNAME:  "$CI_REGISTRY_USER"
    SYFT_REGISTRY_AUTH_PASSWORD:  "$CI_REGISTRY_PASSWORD"
    SYFT_FILE_CONTENTS_SKIP_FILES_ABOVE_SIZE: 20000000
    GRYPE_REGISTRY_AUTH_AUTHORITY: "$CI_REGISTRY"
    GRYPE_REGISTRY_AUTH_USERNAME:  "$CI_REGISTRY_USER"
    GRYPE_REGISTRY_AUTH_PASSWORD:  "$CI_REGISTRY_PASSWORD"
    GRYPE_DB_CACHE_DIR: "./.cache/grype/"
    GRYPE_EXCLUDE: "./.cache"
    TRIVY_USERNAME: "$CI_REGISTRY_USER"
    TRIVY_PASSWORD: "$CI_REGISTRY_PASSWORD"
    TRIVY_AUTH_URL: "$CI_REGISTRY"
    TRIVY_NO_PROGRESS: "true"
    TRIVY_CACHE_DIR: "./.cache/trivy/"
    TRIVY_VERSION: "v0.43.1"
    FULL_IMAGE_NAME: "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
    IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
  allow_failure: true
  cache:
    key: $CI_JOB_NAME
    paths:
      - ./.cache/
  before_script:
    - apk add --update-cache curl openssl docker-cli
    - mkdir -p ./.out
    - export TRIVY_VERSION=${TRIVY_VERSION:-v0.43.1}
    - curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
    - curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin
    - curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin #${TRIVY_VERSION}
    - curl -sSL -o /tmp/trivy-gitlab.tpl https://github.com/aquasecurity/trivy/raw/main/contrib/gitlab.tpl
    - curl -sSL -o /tmp/trivy-codequality.tpl https://github.com/aquasecurity/trivy/raw/main/contrib/gitlab-codequality.tpl
    - curl -sSL -o /tmp/trivy-html.tpl https://github.com/aquasecurity/trivy/raw/main/contrib/html.tpl
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - echo "--- SYFT ---"
    - time syft packages registry:$FULL_IMAGE_NAME --scope all-layers
    - time syft packages dir:./ --scope all-layers -o table=.out/syft_sbom.txt -o spdx-json=.out/syft_sbom_spdx.json
    - echo "--- GRYPE ---"
    - time grype db update
    - time grype dir:./ --add-cpes-if-none -o table > .out/grype_cve_table.txt
    - time grype dir:./ --add-cpes-if-none -o json > .out/grype_cve_json.json
    - time grype dir:./ --add-cpes-if-none -o cyclonedx > .out/grype_cyclonedx.xml  ## this one is ok
    - time grype sbom:.out/syft_sbom_spdx.json --add-cpes-if-none --fail-on critical --only-fixed
    - time grype dir:./ --add-cpes-if-none
    - time grype dir:./ --add-cpes-if-none --only-fixed
    - time grype build/$CONTAINER_NAME --add-cpes-if-none -o table #--fail-on critical --only-fixed
    - time grype registry:$FULL_IMAGE_NAME --add-cpes-if-none --fail-on critical --only-fixed
    - echo "--- TRIVY ---"
    - trivy --version
    - time trivy image --clear-cache
    - time trivy image --download-db-only
    - time trivy image --exit-code 0 --format template --template "@/tmp/trivy-gitlab.tpl" -o .out/trivy_cve.json $IMAGE
    - time trivy image --exit-code 0 --format template --template "@/tmp/trivy-codequality.tpl" -o .out/trivy_codequality.json $IMAGE
    - time trivy image --exit-code 0 --format template --template "@/tmp/trivy-gitlab.tpl" -o .out/gl-container-scanning-report.json $IMAGE
    - time trivy image --exit-code 0 --format template --template "@/tmp/trivy-html.tpl" -o .out/trivy_cve.html $IMAGE
    - time trivy sbom --format spdx -o ".out/trivy_sbom_fs_spdx.spdx" --type fs .
    - time trivy sbom --format spdx-json -o ".out/trivy_sbom_spdxjson.json" --type image "$FULL_IMAGE_NAME"
    - time trivy image --format cyclonedx -o ".out/trivy_sbom_cyclonedx.json" "$FULL_IMAGE_NAME"
    - time trivy image --exit-code 0 "$FULL_IMAGE_NAME"
    - time trivy image --exit-code 0 --severity CRITICAL "$FULL_IMAGE_NAME"
    - time trivy image --exit-code 1 --severity CRITICAL --ignore-unfixed "$FULL_IMAGE_NAME"
  artifacts:
    reports:
      dependency_scanning: .out/grype_cve.json
      container_scanning: .out/gl-container-scanning-report.json
      codequality: .out/trivy_codequality.json
    name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}
    paths:
      - .out/
  

The result of Grype 's in syft_sbom_spdx.json work looks something like this:

    JSON
   
   #grype sbom:.out/syft_sbom_spdx.json --add-cpes-if-none --fail-on critical --only-fixed
NAME                         INSTALLED  FIXED-IN  TYPE       VULNERABILITY        SEVERITY
go.mongodb.org/mongo-driver  v1.1.2     1.5.1     go-module  GHSA-f6mq-5m25-4r72  Medium

The result of Trivy in trivy_codequality.json work looks something like this:

    JSON
   
 

   #trivy image --exit-code 0 "$FULL_IMAGE_NAME"
2022-06-07T13:18:57.228Z  INFO  Detected OS: alpine
2022-06-07T13:18:57.229Z  INFO  Detecting Alpine vulnerabilities...
2022-06-07T13:18:57.230Z  INFO  Number of language-specific files: 1
2022-06-07T13:18:57.230Z  INFO  Detecting gobinary vulnerabilities...
Total: 2 (UNKNOWN: 1, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)
┌─────────────────────────────┬────────────────┬──────────┬───────────────────┬───────────────┬──────────────────────────────────────────────────────────────┐
│           Library           │ Vulnerability  │ Severity │ Installed Version │ Fixed Version │                            Title                             │
├─────────────────────────────┼────────────────┼──────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ go.mongodb.org/mongo-driver │ CVE-2021-20329 │ MEDIUM   │ v1.1.2            │ 1.5.1         │ mongo-go-driver: specific cstrings input may not be properly │
│                             │                │          │                   │               │ validated                                                    │
│                             │                │          │                   │               │ https://avd.aquasec.com/nvd/cve-2021-20329                   │
├─────────────────────────────┼────────────────┼──────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ golang.org/x/text           │ CVE-2021-38561 │ UNKNOWN  │ v0.3.3            │ 0.3.7         │ Due to improper index calculation, an incorrectly formatted  │
│                             │                │          │                   │               │ language tag can cause...                                    │
│                             │                │          │                   │               │ https://avd.aquasec.com/nvd/cve-2021-38561                   │
└─────────────────────────────┴────────────────┴──────────┴───────────────────┴───────────────┴──────────────────────────────────────────────────────────────┘
  

Conclusion

Vulnerability scanning plays a crucial role in strengthening the security of software applications and ensuring a proactive approach to addressing vulnerabilities. By integrating vulnerability scanners into your CI pipeline for container images and source code, you can detect vulnerabilities early, maintain compliance, and foster a secure coding culture. Leveraging automation and comprehensive scanning tools allow teams to efficiently monitor and continuously improve the security of their software, ultimately delivering safer applications to end-users.

Contextual design JSON Software development Vulnerability Cache (computing) security

Opinions expressed by DZone contributors are their own.

Related

Trending