Secrets Management: Using Vault for Accessing the Cloud Infrastructure

Introduction

In the current IT world, there has been a need for having secured connectivity and having many private objects, which need to be accessible only for a limited set of applications or services. We call this orchestration “Secrets Management.” There are many tools currently available in the market that caters to this need. Some of them are inbuilt with the cloud-like Secrets Manager for AWS or Docker Secrets or Vaults. Today, in this post, we will discuss Managing Secrets by Vault and Hashicorp. There are many features that are exposed by Vault in regards to implementing and securing the application authorization and authentication.

We will discuss how dynamic secrets can be generated by Vault using AWS IAM Policies and how to send them to an application via API-based calls. An advantage of dynamic secrets is that they are generated when they are accessed. Dynamic secrets do not exist until they are read, so there is no risk of someone stealing them or another client using the same secrets. Because Vault has built-in revocation mechanisms, dynamic secrets can be revoked immediately after use, minimizing the amount of time the secret existed.

Implementation

We will demonstrate the configuration of AWS inside Vault using the secrets engine plugin and generate dynamic credentials with pre-configured IAM policies embedded inside the vault to securely access AWS SDK.

Technology stacks used for this post:

• Java
• AWS SDK – Dynamo DB
• AWS IAM – Policy
• Vault

Vault Configuration

Enabling Secrets Engine

We will show this post with vault hosted in the local machine. The configuration of hosting and installing vault is not the scope of this post. Once we login to Vault Servers using Token or other authentication methods, we can enable a secret engine using the AWS plugin.

We select the AWS plugin in the vault (which can be done using API commands) and add the credentials of the AWS account inside the secrets engine in the following page.

AWS Secured Configuration

In the above page, we can add credentials from our AWS Account and have the additional configuration (not mandatory) about regional-based details or any other IAM endpoint appended the secret engine. Once the AWS and Vault successfully integrate, we have the option of creating role based on them. Role is the dynamic secrets that can be configured with the policies on top of them to allow/ deny usage of any AWS services. We will be using simple Allow All Policy in the example.

Adding AWS Role Inside Vault

This Role will create dynamic secrets with time validity and can be used inside any third-party application to securely access AWS SDK’s.

Dynamic Secrets Generation

The Above Credentials are generated and can be consumed into applications externally or using API-based invocation. There are also other types of credentials, like Federated Token or Assumed Role, that can also be generated by the same approach. We will show using a Spring Boot application where the credentials are obtained by an API and demonstrate CRUD operations on DynamoDB. We have created a small boot application and added a RestTemplate to invoke the API of the vault.

Spring – Vault Integration

Rest API – Vault Invocation

We make a GET call towards the installed vault server with the token inside the X-Vault-Token header and /v1/aws/creds/{role_name}. The Role Name is the same one created inside the vault with IAM Policies and customized service access.

Once Vault Response contains the access_key and secret_key dynamically and is provided inside the AWS SDK to create a connection with Spring Boot and AWS ecosystem...

Adding Dynamic Secrets to AWS SDK

The Program then continues to perform CRUD operations inside the DynamoDB service. We can modify various operations on any of the services based on the Policies attached to the Role.

Conclusion

Finally, we are done with the configuration. In the above mechanism, we have demonstrated the use of vault for using AWS SDK’s without exposing the IAM credentials to the Spring Boot application. There are many other useful features of vault, like PKI Certificate Secret Storage, Key-Value Storage, or SSH storage integration. The Documentation of Vault also contains the details of setting up the vault inside the cluster or using terraform to automate the infrastructure.

Reference

• https://www.vaultproject.io/
• https://spring.io/blog/2016/06/24/managing-secrets-with-vault
• https://aws.amazon.com/sdk-for-java/