How To Use ggshield To Avoid Hardcoded Secrets [Cheat Sheet Included]
GitGuardian’s ggshield CLI tool can help you keep your secrets away from your repos and pipelines. Download our handy cheat sheet to quickly become proficient in our CLI tool. This article also includes a cheat sheet.
Join the DZone community and get the full member experience.Join For Free
Most developers love working in the terminal, tying together all sorts of tools with command line interfaces, CLIs, via scripting. Working with CLIs is powerful, but it can be challenging to initially learn all the ways a tool can help you do your work. While the only real way to learn any tool is by using it, one time-tested method to get over the learning curve is to keep a short list of common commands, as well as concept recaps, on hand for using the tools.
At the same time, it is pretty common to adopt a tool for one or two specific functions, without investigating what other commands a tool offers. For example, just think about how many Git commands you use regularly out of the 164 currently available commands. Getting a holistic view of how a command line tool is structured and the possible commands can help you make better use of the platform.
With the goal of making ggshield easier to learn while giving a compact, yet comprehensive overview of available commands, we are proud to present our new ggshield cheat sheet.
The first thing to understand about any CLI tool is how the commands are structured. Just as spoken languages follow rules of grammar to make sense, CLI tools follow a specified pattern, called syntax. For ggshield, the syntax is:
ggshield [COMMAND] [SUBCOMMAND] [OPTIONS]
Of course, the first part is
ggshield, invoking the tool itself. Next, you need to state what command you want ggshield to run. Subcommands and options give the application the fine-grained details it needs to accomplish your requested task. In addition to the specific options each subcommand can offer, there are a couple of global options that are always available.
All ggshield commands offer usage assistance through the option
--help or just
ggshield -h will print the syntax for ggshield commands as well as show a list of the available high-level commands.
--help which can go at the end of any command,
--json needs to go after the subcommand but before any options. For example, to return the output of
ggshield secrets scan path . as JSON, use:
ggshield secrets scan --json path .
Connecting to the GitGuardian Platform
ggshield leverages the GitGuardian platform to perform scans looking for more than 350 types of secrets. As you are getting set up with ggshield, after installing the tool, the next step is to authenticate against the platform. This can be done with the
auth command and
ggshield auth login
Running that command without any other options will open a new browser window, allowing you to log into your workspace, or create a new account, and GitGuardian will automatically generate a personal access token and store it in your configuration.
You can also manually set your authentication token, redirect to your enterprise SSO site, or even set a time to live for your authentication token. You can read more about those options in the documentation.
While you can stay logged in forever, you might want to log out sometimes. Removing your authentication token is very simple:
ggshield auth logout
Secret Scan and Ignore
The ggshield secret command is the main command you will likely use day to day. There are currently 2 subcommands for the secret command: scan and ignore. The
secret scan commands require some additional options to specify exactly what you want the tool to accomplish.
While ggshield is powerful, it is good to know the current limits to its superpowers. GitGuardian will not scan individual files over 1MB in size or any files with binary extensions. Also, know that CLI commands can sometimes consume multiple API calls. For example, a repo scan can result in a large number of API calls, depending on the size of the repo; which is a good reason to keep an eye on your quota.
Secret Scan Repo
As the first part of our company's name implies, we are experts in searching Git repositories. The
secret scan repo command lets you search repos either locally, or on any URL you can access. If you are specifying a URL you will need to make sure it ends in
.git. For example, if you want to run a secrets scan on the public demo repo GitGuardian/sample_secrets, you would use:
ggshield secret scan repo https://github.com/GitGuardian/sample_secrets.git
Note: Running this query will produce a lot of results, as this repository is an example of how not to store your credentials in your code.
Secret Scan Path
secret scan repo is useful for locally cloned repositories, ggshield can also scan any files, even outside of version control. This can be used as you are creating a new project as a way to ensure no secrets make it near the initial commit. For example, to check for secrets in the directory you are currently working in, use:
ggshield secret scan path --recursive .
If you want to scan a folder containing subfolders, you will need to add the
Secret Scan Commit-Range
One major issue with hardcoding and committing secrets is they are there forever by default. Even if you later remove the API key, password, or other credentials, from your latest commits, any previous commits and secrets will still be present in your permanent project history.
ggshield secret scan repo will search through all the commits in all of your repo's branches, you might want to focus your search on just between certain refs. To specify the range, ggshield uses a
... notation, a notation also used by Git. Any commit-ish ref can be used, such as a full commit ID, branch name, tag, or HEAD. Leveraging that last option, we can quickly look at a commit range between the most recent and the prior 3 commits you would use:
ggshield secret scan commit-range HEAD...HEAD~3
Secret Scan Archive
Archiving your Git repos makes your code even easier to package, copy and distribute. It also means any secrets included in an archive are going to be in all future copies and can end up in all sorts of unexpected places. Fortunately, ggshield gives you a way to make sure any
.zip, .tar, .tar.gz, .tar.bz2, and .tar.xz archives are free of secrets.
To scan an archive, use:
ggshield secret scan archive path/to/archive
Secret Scan CI, Docker, And Pypi
GitGuardian is built for DevOps generation. Modern tools like CI environments, containerization, and dependency management are all areas that need security scanning. These three commands are aimed to help at various points in the SDLC.
Before you add that Python dependency to your project, test to see if there are any suspicious secrets lurking about it. Just specify the Python Package Index name and ggshield will pull it down and scan it for you. To scan a PyPI package use:
ggshield secret scan pypi PACKAGE_NAME>
Containers can make it extremely easy to share your work with other developers. Using Docker can mean the end of "it worked on my machine". However, packaging up a whole container with
docker save means you might be sharing secrets in the exported filesystem and manifest. With ggshield, you can quickly scan those files using:
ggshield secret scan docker <IMAGE_NAME>
Unlike all our other commands you have seen so far, scanning a CI requires you to be working within a pipeline. This command can help you quickly test for secrets before build steps are triggered. To scan the set of commits pushed that triggered the CI pipeline, use:
ggshield secret scan ci
While it is very important to catch your secrets before you ship them to production, it is also important to eliminate false positives. While GitGuardian scanning is high precision, we also allow you to fine-tune for exceptions. If a secret is uncovered in a secrets scan of a repo or a path, you can add that term to the ignore list, stored in your ggsheild config. To ignore a found secret from showing up in future scans, use:
ggshield secret ignore –last-found
While you can manually run a ggshield secret scan to find any credentials in your files, you can also run it every time you make a commit or push by leveraging Git's built-in automation platform, Git Hooks.
Automating scanning relies on commands
ggshield secret scan pre-commit and
ggshield secret scan pre-push. These two commands are built to quickly check just the tracked changes as you are working with Git.
You can quickly add ggshield secret scan pre-commit to your local
.git/hooks folder by using the command:
ggshield install –mode local –hook-type pre-commit
If you are already using Git Hooks, then you will most likely want to append the existing file instead of writing a new one. You can do this with the
-a option. For example, if you wanted to add the pre-push hook command globally, use:
ggshield install –mode global –hook-type pre-push -a
While you can't install this hook in your local Git client with
ggshield install, you can use the command
ggshield secret scan pre-receive in your CI/CD pipelines to check for secrets before the commit has been applied to the receiving server.
Infrastructure as Code Scan
We are happy to introduce the newest command and capability for the ggshield, the ability to scan for common configuration vulnerabilities in your IaC files. GitGuardian will perform over 70 checks against your IaC folder, either locally or in a CI environment. To perform a scan using:
ggshield iac scan path_to_main_iac_folder
You can customize your IaC scans in a number of different ways. For example, inserting
--exit-zero right after
scan will always return a 0 (non-error) status code and
--ignore-path will ignore any specific folders or files. To help you eliminate noise from your scan, you can use
--minimum-severity to set the lower threshold for alerts based on the severity of LOW to CRITICAL. Read more about these options and about using
ggshield iac in the documentation.
Like any other CLI tool, ggshield relies on configuration files to manage settings. With ggshield there are three levels of configuration, loaded in this order:
As with Git and many other tools, each configuration file loaded will override any settings defined by previously loaded files.
The global level configuration is set at the user home directly level,
~/.gitguardian.yml on Linux and macOS, and
%USERPROFILE%\.gitguardian on Windows. The local-level configuration is pulled from the
.gitgardian.yml file that lives at the root of your project.
GitGuardian allows you to define your own configuration files that can live anywhere you want and will be evaluated instead of your global or local-level config. We call this CLI-level configuration. To call a custom CLI configuration, use the option
--config-path in any command. For example, to call a CLI config located at
~/Desktop/only_config.yaml while scanning a path, use:
ggshield --config-path=~/Desktop/only_config.yaml scan path -r .
If you want to build a configuration from an example, you can find a sample config file on GitHub example..
ggshield config list command will print the list of configuration keys and values, including
workspace_id, admin dashboard URL, token,
token_name, and token expiration if set. These are not stored in your global or local configuration
gitguardian.yml files. These are not items you would typically want to share.
While the GitGuardian platform is very robust and highly available, sometimes you need to troubleshoot why a tool is not delivering the expected results. As with any tool connecting to a platform API, ggshield provides a way to test if the API endpoints are available.
While you can, and should, run ggshield for free on your repositories, you should also be aware there is a monthly limit on the number of API calls you can make. By default, we grant 1,000 calls/month on our free plans and 10,000 calls/month for our paying customers. Those quotas can be fine-tuned upon request. To see how your available API call quota, use:
Make ggshield Part of Your Workflow
GitGuardian's ggshield can help you quickly find any secrets in your repos, local files, archives, and commits. The CLI tool can also help you avoid sharing your secrets in your CI pipeline, Docker files, and even dependencies. Automate your workflow with the help of Git Hooks and ggshield install. And now we are proud to help you find and fix vulnerabilities in your Infrastructure as Code files.
While there are a lot of options and parameters you can use, at its heart, ggshield is a very straightforward tool, designed with the developer in mind, with a mission to make it easier for you to find and fix secrets sprawl and IaC misconfigurations.
Published at DZone with permission of Dwayne McDaniel. See the original article here.
Opinions expressed by DZone contributors are their own.