DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Docker Model Runner: Streamlining AI Deployment for Developers
  • A Guide to Container Runtimes
  • Gemma 3: Unlocking GenAI Potential Using Docker Model Runner
  • Docker vs Kubernetes: Which to Use and When?

Trending

  • Understanding and Mitigating IP Spoofing Attacks
  • The Role of Functional Programming in Modern Software Development
  • Enhancing Security With ZTNA in Hybrid and Multi-Cloud Deployments
  • How to Format Articles for DZone
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. What a Mysterious Bug Taught Us About How Docker Stores Registry Credentials

What a Mysterious Bug Taught Us About How Docker Stores Registry Credentials

Where does Docker store registry credentials?

By 
Ethan J Jackson user avatar
Ethan J Jackson
·
Jul. 06, 20 · Presentation
Likes (4)
Comment
Save
Tweet
Share
8.3K Views

Join the DZone community and get the full member experience.

Join For Free

We recently ran into a mysterious bug that required hours of digging into the arcane details of Docker’s registry credentials store to figure out. Although in the end, the fix turned out to be easy, we learned a thing or two along the way about the design of the credentials store and how, if you’re not careful, it can be configured insecurely.

Blimp sometimes needs to pull private images from a Docker registry in order to boot those images in the cloud. This typically works fine, but unfortunately, when some users started Blimp, they were getting the following error message:

Plain Text
xxxxxxxxxx
1
 
1
Get https://1234.dkr.ecr.us-east-1.amazonaws.com/v2/blimp/blimp/manifests/v0.1: no basic auth credentials


At first, we were completely baffled by this cryptic message and had no clue it was related to our handling of credentials. To understand how we figured it out, first you need to know a little about how modern Docker credentials are handled.

Docker’s External Credentials Store

The recommended way to store your Docker credentials is in an external credentials store. In your Docker config file, which is usually located at ~/.docker/config.json, there are two fields you can use to configure how Docker gets and stores credentials: credsStore and credHelpers.

credsStore tells Docker which helper program to use to interact with the credentials store. All helper programs have names that begin with docker-credential- – the value of credsStore is the suffix of the helper program.

For example, if you work on a Mac laptop, you might decide to use the Mac OS keychain. The name of the helper program to use the keychain is docker-credential-osxkeychain. So your config.json would include the following:

JSON
xxxxxxxxxx
1
 
1
{
2
  "credsStore": "osxkeychain"
3
}


If you want to see what credentials Docker currently has for you, you can use list. For example:

Plain Text
xxxxxxxxxx
1
 
1
docker-credential-osxkeychain list


The result is a list of pairs of servers and usernames. For example:

JSON
xxxxxxxxxx
1
 
1
{
2
  "http://quay.io":"kklin",
3
  "https://index.docker.io":"kevinklin"
4
}


You may also notice credHelpers in your config.json. These helpers are similar to credsStore, but are used to generate short lived credentials. For example, if you use gcr, gcloud installs a credHelper that uses your Google login to get tokens. This way, Docker never has your Google credentials directly – the docker-credential-gcloud acts as a middleman between Docker and your Google credentials.

Once again, here’s the error message our users were getting:

Plain Text
xxxxxxxxxx
1
 
1
Get https://1234.dkr.ecr.us-east-1.amazonaws.com/v2/blimp/blimp/manifests/v0.1: no basic auth credentials


We were able to run the docker-credential-osxkeychain list and get commands to see the credentials for 1234.dkr.ecr.us-east-1.amazonaws.com, so why were we getting an error that there weren’t any credentials??

In the Beginning: Docker Stores Your Registry Password in Your Config File

It turns out that external credentials stores weren’t added to Docker until version 1.11, in 2016. Before 1.11, Docker stored credentials via a config field called auths. This field is stored in the same file as the credStore: ~/.docker/config.json.

Whenever you logged into a registry, Docker would set the value of auths to your password. For example, your config file might contain the following:

JSON
xxxxxxxxxx
1
10
 
1
{
2
    "auths": {
3
        "https://index.docker.io/v1/": {
4
            "auth": "YW11cmRhY2E6c3VwZXJzZWNyZXRwYXNzd29yZA=="
5
        },
6
        "localhost:5001": {
7
            "auth": "aGVzdHVzZXI6dGVzdHBhc3N3b3Jk"
8
        }
9
    }
10
}


What we learned the hard way is that there’s a quirk with Docker’s login command. When you log in using docker login, Docker adds an entry via the credsStore and in auths, using slightly different server names. Your credentials are properly stored in the credentials store, but the entry in auths doesn’t contain the username or password. The result looks something like this:

JSON
xxxxxxxxxx
1
 
1
{
2
"auths": {
3
  "https://index.docker.io/v1/": {}
4
}


The problem is that Blimp grabs credentials from both auths and credsStore. So it was passing two copies of the credentials to the Docker image puller – one with the correct username and password, and one without the password at all.

Unfortunately, Docker preferred the https:// version of the credential, and attempt to pull the image with the empty credential. Thus, the no basic auth credentials error.

Once we figured out that the problem was that an empty duplicate entry was getting added to the insecure store, it was easy to fix the problem. All we needed to do was add an if statement to skip empty credentials:

Go
xxxxxxxxxx
1
21
 
1
addCredentials := func(authConfigs map[string]clitypes.AuthConfig) {
2
    for host, cred := range authConfigs {
3
        // Don't add empty config sections.
4
        if cred.Username != "" ||
5
            cred.Password != "" ||
6
            cred.Auth != "" ||
7
            cred.Email != "" ||
8
            cred.IdentityToken != "" ||
9
            cred.RegistryToken != "" {
10
            creds[host] = types.AuthConfig{
11
                Username:      cred.Username,
12
                Password:      cred.Password,
13
                Auth:          cred.Auth,
14
                Email:         cred.Email,
15
                ServerAddress: cred.ServerAddress,
16
                IdentityToken: cred.IdentityToken,
17
                RegistryToken: cred.RegistryToken,
18
            }
19
        }
20
    }
21
}


A Potential Docker Credentials Security Risk

In the process of uncovering this bug, we noticed a potential security risk that you may not be aware of. As we learned, it’s best practice to use an external store to store your external registry credentials. However, depending on how and when you installed Docker it’s possible you could still be using the legacy auths method. If you are, your ~/.docker/config.json might look something like this:

JSON
xxxxxxxxxx
1
10
 
1
{
2
    "auths": {
3
        "https://index.docker.io/v1/": {
4
            "auth": "YW11cmRhY2E6c3VwZXJzZWNyZXRwYXNzd29yZA=="
5
        },
6
        "localhost:5001": {
7
            "auth": "aGVzdHVzZXI6dGVzdHBhc3N3b3Jk"
8
        }
9
    }
10
}


This may look reasonable secure, the passwords appear to be a garbled bunch of gibberish. Surely those passwords are encrypted, right?

Guess again. All Docker did was encode the passwords using base64. And as David Rieger pointed out on Hacker Noon, base64

may look like encryption on first glance, but it’s not. Base64 is a scheme for encoding, not encryption. You can simply copy the base64 string and convert it to ASCII in a matter of seconds.

That seemingly secure password of aGVzdHVzZXI6dGVzdHBhc3N3b3Jk? All you need to do to read the password is base64 decode it:

Shell
xxxxxxxxxx
1
 
1
$ echo aGVzdHVzZXI6dGVzdHBhc3N3b3Jk| base64 -D
2
hestuser:testpassword


The Moral of Our Story: Double Check Your Docker Credentials’ Security

So that’s the bad news: if Docker config file isn’t properly set up, Docker is storing your credentials password in plain text.

The good news is that it’s easy to fix the problem.

All you and your team members need to do is take a quick look at ~/.docker/config.json. If it contains an auths password, get rid of it and switch over to using a credentials store. To do so, just download the appropriate docker-credential- helper for your system, and update the credsHelper field in ~/.docker/config.json.

Hope that helps!

---

Read the Top 5 common mistakes when writing Docker Compose.

Docker (software)

Published at DZone with permission of Ethan J Jackson. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Docker Model Runner: Streamlining AI Deployment for Developers
  • A Guide to Container Runtimes
  • Gemma 3: Unlocking GenAI Potential Using Docker Model Runner
  • Docker vs Kubernetes: Which to Use and When?

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!