How to Vaults and Wallets for Simple, Secure Connectivity

This is the third in a series of blogs on data-driven microservices design mechanisms and transaction patterns with the Oracle converged database. The first blog illustrated how to connect to an Oracle database in Java, JavaScript, Python, .NET, and Go as succinctly as possible. The second blog illustrated how to use that connection to receive and send messages with Oracle AQ (Advanced Queueing) queues and topics and conduct an update and read from the database using all of these same languages.  The goal of this third blog is to provide details on how to secure connections in these same languages as well as convenience integration features that are provided by microservice frameworks, specifically Helidon and Micronaut.

When making secure connections to Oracle databases there are two items to consider, the wallet and the password. We will discuss and provide examples of both in this blog.

One-way TLS or Mutual TLS With Wallet

Oracle Wallet is a container that stores authentication and signing credentials, providing mutual TLS authentication (all communications between the client and the server are encrypted), and is a requirement for connecting to the Oracle Autonomous Databases unless One-way TLS is used.

It is now also possible to connect to ADB (Oracle Autonomous databases) using one-way TLS which does not require a wallet. This is enabled in three steps:

1. If the instance is configured to operate over the public internet, then one or more Access Control Lists (ACLs) must be defined on the serverside (under Network section of the database details page of the OCI console).
2. "Require mutual TLS" must be set to false on the serverside (again, under Network section of the database details page of the OCI console).
3. "?ssl_server_cert_dn=<DN>" must be appended to the connection URL used by the client.

Documentation with background information, screenshots of the corresponding OCI consoles, etc. can be found here.

Wallets can optionally also be used to store one or more password credentials.

• These credentials are added using the mkstore tool.
• This removes the need to provide a password explicitly in order to connect as it is part of the wallet, and if there is only one credential stored in the wallet it removes the need to provide a username as well.
• This provides flexibility and potential convenience, however, it is common to maintain wallet and password security separately particularly in a microservices environment.

The location of the wallet can be indicated via the TNS_ADMIN environment variable or, in the case of Java, also provided as an URL property.  The WALLET_LOCATION can also be overridden in the sqlnet.ora file.  See first blog for details.  The wallet can also be set programmatically using setSSLContext(SSLContext sslContext). Helidon and Micronaut have convenience features that automatically download ADB wallets to create datasources.

Passwords

Passwords for microservices are generally stored in either Kubernetes secrets or a vault of some form such as Hashicorp Vault or OCI Vault.  Note that vaults are a concept, not a standard, and so there is no common API per se though usage is expectedly similar.

Kubernetes secrets are not as secure as vault secrets as the information in them is merely base64 encoded and they must be provided to the microservice runtime environment in one form or another (as environment variables, mounted files, etc.) whereas the vault provides a convenient approach for retrieving passwords at runtime from within the microservice itself. This, in addition, provides a natural way to accommodate the rotation of passwords (another security best practice) to be picked up dynamically by the microservice.

Both HashiCorp and OCI Vault integrations exist in Helidon and Micronaut, though OCI Vault is considerably easier to set up and manage.

*Note that the vaults mentioned here are not to be confused with the Oracle Database Vault which implements data security controls within Oracle Database to restrict access to application data by privileged users.

OCI Vault documentation is simple and straightforward and the basic roles and flow are shown in the following diagram.

Admins create databases and vault secrets containing the passwords for the databases and provide the OCIDs (OCI resource Ids) of the secrets (eg "ocid1.vaultsecret.oc1.iad.aafafaai5grzfemrbwa") to the application.

The application retrieves the password/secret from the vault and uses it to access the database.

Applications authenticate in order to access OCI Vault via instance principal, OCI config, etc. (discussed in the next section on code snippets).

Setting up the vault and secrets involves three easy steps:

1. Create dynamic-group(s) for the comparment(s) (or instance(s)) that will need vault access. (under Identity and Security/Dynamic Groups section of the OCI console).
   
2. Create policy(ies) for managing and accessing Vault secrets, keys, etc. and assign it to the appropriate group(s) (under Identity and Security/Policies section of the OCI console).
   • Following the principle of least privilege, there should be separate dynamic groups and policies as appropriate. For example...
     • For admins...
       • allow dynamic-group my-group to manage secret-family in compartmentxyz
       • allow dynamic-group my-group to manage vault in compartmentxyz
       • allow dynamic-group my-group to manage keys in compartmentxyz 
     • For users...
       • allow dynamic-group my-secret-group to read secret-family in compartment my-compartment where target.secret.name = 'atpOrderDBPw'
3. Create encryption key(s) for secrets. (under Identity and Security/Vault section of the OCI console).
    
4. Create secret(s). (again, (under Identity and Security/Vault section of the OCI console).
   

Code Snippets

The following are examples for all languages mentioned as well as Helidon and Micronaut framework features.

As always full source and working examples can be found at https://github.com/oracle/microservices-datadriven and can be run as part of the Simplifying Microservices with converged Oracle Database Workshop at https://bit.ly/simplifymicroservices

In order to create a secure OCI client connection to retrieve Vault secrets, an AuthenticationDetailsProvider, of which there are a number of types, is used.

The examples use the InstancePrincipalsAuthenticationDetailsProvider which will implicitly use the instance principal that is associated with the Kubernetes pod of the microservice to generate service tokens used for signing.  As such this requires no additional configuration.

This ConfigFileAuthenticationDetailsProvider can be used instead and is created by providing an oci config file (generally found at ~/.oci/config ) which can be useful for local development, if the program/microservice is being run outside of Kubernetes, for example.  An example showing this as an option can be seen in the Java snippet (it is the same for all languages and thus is not repeated).

The OCI SDKs provide a number of different clients for various services. It is possible to use either the VaultsClient or SecretsClient of the SDKs to retrieve the password secrets.  The SecretsClient is used in the examples provided as it is somewhat more appropriate and direct (we only needs secrets, not other Vault operations), however, we provide examples using both SecretsClient in the Go example to give you the idea.

Generally, the secret will be base64encoded and so to be complete the samples also finish by decoding the secret/password value retrieved from the vault.

We'll provide just the basic facts you need... packages, imports, and source (and in the case of frameworks, config as well).

Java

static String getSecreteFromVault(boolean isInstancePrincipal, String regionIdString, String secretOcid) throws IOException {
    System.out.println("OCISDKUtility.getSecretFromVault isInstancePrincipal:" + isInstancePrincipal);
    SecretsClient secretsClient;
    if (isInstancePrincipal) {
        secretsClient = new SecretsClient(InstancePrincipalsAuthenticationDetailsProvider.builder().build());
    } else {
        secretsClient = new SecretsClient(new ConfigFileAuthenticationDetailsProvider("~/.oci/config", "DEFAULT"));
    }
    secretsClient.setRegion(regionIdString);
    GetSecretBundleRequest getSecretBundleRequest = GetSecretBundleRequest
            .builder()
            .secretId(secretOcid)
            .stage(GetSecretBundleRequest.Stage.Current)
            .build();
    GetSecretBundleResponse getSecretBundleResponse = secretsClient.getSecretBundle(getSecretBundleRequest);
    Base64SecretBundleContentDetails base64SecretBundleContentDetails =
            (Base64SecretBundleContentDetails) getSecretBundleResponse.getSecretBundle().getSecretBundleContent();
    byte[] secretValueDecoded = Base64.decodeBase64(base64SecretBundleContentDetails.getContent());
    return new String(secretValueDecoded);
}

Python

import oci
import base64

#...
 

signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()
secrets_client = oci.secrets.SecretsClient(config={'region': region_id}, signer=signer)
secret_bundle = secrets_client.get_secret_bundle(secret_id = vault_secret_ocid)
logger.debug(secret_bundle)
base64_bytes = secret_bundle.data.secret_bundle_content.content.encode('ascii')
message_bytes = base64.b64decode(base64_bytes)
db_password = message_bytes.decode('ascii')

JavaScript

const identity = require("oci-identity");
const common = require('oci-common');
const secrets = require('oci-secrets');

//...

async function getSecret() {
  const provider = await new common.InstancePrincipalsAuthenticationDetailsProviderBuilder().build();
  try {
      const secretConfig = {
       secretInfo: {
         regionid: process.env.OCI_REGION,
         vaultsecretocid: process.env.VAULT_SECRET_OCID,
         k8ssecretdbpassword: process.env.dbpassword
       }
      };
      if  (secretConfig.secretInfo.vaultsecretocid == "") {
        pwDecoded = process.env.dbpassword;
      } else {
        console.log("regionid: ", secretConfig.secretInfo.regionid);
        console.log("vaultsecretocid: ", secretConfig.secretInfo.vaultsecretocid);
        const client =  new secrets.SecretsClient({
            authenticationDetailsProvider: provider
        });
        const getSecretBundleRequest = {
                secretId: secretConfig.secretInfo.vaultsecretocid
            };
        const getSecretBundleResponse = await client.getSecretBundle(getSecretBundleRequest);
        const pw = getSecretBundleResponse.secretBundle.secretBundleContent.content;
        let buff = new Buffer(pw, 'base64');
        pwDecoded = buff.toString('ascii');
      }
  } catch (e) {
    throw Error(`Failed with error: ${e}`);
  }
}

.NET

<PackageReference Include="OCI.DotNetSDK.Common" Version="29.0.0" />
<PackageReference Include="OCI.DotNetSDK.Secrets" Version="29.0.0" />

using Oci.SecretsService.Responses;
using Oci.SecretsService;
using Oci.Common;
using Oci.Common.Auth;
using Oci.SecretsService.Models;

//...

     
     public String getSecretFromVault() {
         var response = getSecretResponse(vaultSecretOCID, ociRegion).GetAwaiter().GetResult();
         byte[] data = System.Convert.FromBase64String(((Base64SecretBundleContentDetails)response.SecretBundle.SecretBundleContent).Content);
         return System.Text.ASCIIEncoding.ASCII.GetString(data);
     }

     public static async Task<GetSecretBundleResponse> getSecretResponse(string vaultSecretOCID, string ociRegion)
     {
         var getSecretBundleRequest = new Oci.SecretsService.Requests.GetSecretBundleRequest
         {
            SecretId = vaultSecretOCID
         };
         var provider = new InstancePrincipalsAuthenticationDetailsProvider();
         try
         {
            using (var client = new SecretsClient(provider, new ClientConfiguration()))
            {
                 client.SetRegion(ociRegion);
                 return await client.GetSecretBundle(getSecretBundleRequest);
            }
         }
         catch (Exception e)
         {
             Console.WriteLine($"GetSecretBundle Failed with {e.Message}");
             throw e;
         }
     }

Go

//Using SecretsClient...
"github.com/oracle/oci-go-sdk/v49/common"
"github.com/oracle/oci-go-sdk/v49/common/auth"
"github.com/oracle/oci-go-sdk/v49/secrets"

func getSecretFromVault() string {
  vault_secret_ocid := os.Getenv("VAULT_SECRET_OCID")
    if vault_secret_ocid == "" {
        return ""
    }
    oci_region := os.Getenv("OCI_REGION") //eg "us-ashburn-1" ie common.RegionIAD
    if oci_region == "" {
        return ""
    }
    instancePrincipalConfigurationProvider, err := auth.InstancePrincipalConfigurationProviderForRegion(common.RegionIAD)
    client, err := secrets.NewSecretsClientWithConfigurationProvider(instancePrincipalConfigurationProvider)
    if err != nil {
        fmt.Printf("failed to create client err = %s", err)
        return ""
    }
    req := secrets.GetSecretBundleRequest{SecretId: common.String(vault_secret_ocid)}
    resp, err := client.GetSecretBundle(context.Background(), req)
    if err != nil {
        fmt.Printf("failed to create resp err = %s", err)
        return ""
    }
    base64SecretBundleContentDetails := resp.SecretBundle.SecretBundleContent.(secrets.Base64SecretBundleContentDetails)
    secretValue, err := base64.StdEncoding.DecodeString(*base64SecretBundleContentDetails.Content)
    if err != nil {
        fmt.Printf("failed to decode err = %s", err)
        return ""
    }
    return string(secretValue)
}

//Using VaultClient...
"github.com/oracle/oci-go-sdk/v49/common"
"github.com/oracle/oci-go-sdk/v49/common/auth"
"github.com/oracle/oci-go-sdk/v49/vault"

func getSecretFromVault() string {
   instancePrincipalConfigurationProvider, err := auth.InstancePrincipalConfigurationProviderForRegion(common.RegionIAD)
   client, err := vault.NewVaultsClientWithConfigurationProvider(instancePrincipalConfigurationProvider)
   if err != nil {
      fmt.Printf("failed to create client err = %s", err)
      return ""
   }
   req := vault.GetSecretRequest{SecretId: common.String(vault_secret_ocid)}
   resp, err := client.GetSecret(context.Background(), req)
   if err != nil {
      fmt.Printf("failed to create resp err = %s", err)
      return ""
   }
   fmt.Println(resp)
   secretValue, err := base64.StdEncoding.DecodeString(resp.Secret.String())
   if err != nil {
      fmt.Printf("failed to decode err = %s", err)
      return ""
   }
   return string(secretValue)
}

Helidon

Helidon is a cloud-native, open‑source set of Java libraries for writing microservices that implements the Eclipse MicroProfile specifications. It is very familiar to Java EE developers in particular.

Datasources can be automatically injected into a microservice via annotation based on configuration. For example, the source may simply have the following...

@Inject
DataSource atpInventoryPDB;

or 

@Inject
@Named("inventorypdb")
PoolDataSource atpInventoryPDB;

It also integrates with ADB to automatically download the wallet used to connect to an Oracle Autonomous Database such that the user only needs to provide the OCID of and a wallet password for the ADB that is to be connected to. Here is an example of such an application.yaml config file...

oracle:
  ucp:
    jdbc:
      PoolDataSource:
        atp:
          connectionFactoryClassName: oracle.jdbc.pool.OracleDataSource
          userName: "ADMIN"
          password: "HelidonATP123"
          serviceName: "helidonatp"
oci:
  atp:
    ocid: "ocid1.autonomousdatabase.oc1.iad.anuwasdfasfdasdfasdfadfcebvb5ehmxlu22xpfwq"
    walletPassword: HelidonTest1

This is documented here with an example here.

Helidon also has integration with HashiCorp Vault and various OCI services including the Vault service and allows injecting of Vault resources based on config that can then be used to access secrets that may be used when creating database connections.

@Inject
VaultResource(OciVault vault,
@ConfigProperty(name = "app.vault.vault-ocid") String vaultOcid,
@ConfigProperty(name = "app.vault.compartment-ocid") String compartmentOcid,
@ConfigProperty(name = "app.vault.encryption-key-ocid") String encryptionKeyOcid,
              @ConfigProperty(name = "app.vault.signature-key-ocid") String signatureKeyOcid)

This is documented here and described well in this blog.

Micronaut

The Micronaut framework is a modern, open-source, JVM-based, full-stack toolkit for building modular, easily testable microservice and serverless applications.  It is very familiar to Spring Boot developers in particular.

Datasources can be automatically injected into a microservice via annotation based on configuration.

It also integrates with ADB to automatically download the wallet used to connect to an Oracle Autonomous Database such that the user only needs to provide the OCID of the ADB that is to be connected to.

This is documented here and described well in this blog.

Micronaut also has integration with HashiCorp Vault and various OCI services including the Vault service which can be configured using a resource/bootstrap-oraclecloud.yaml such as the following...

micronaut:
  config-client:
    enabled: true
oci:
  config:
    profile: DEFAULT 
  vault:
    config:
      enabled: true
    vaults:
      - ocid: ocid1.vault..aaaaaaaatafc2boxebiasdfasdfasdfzwzjxgn3xq24hbqvq
        compartment-ocid: ocid1.compartment.oc1..aaaaaaaatafcasdfasdfasdfwcbacw6qrzwzjxgn3xq24hbqvq

Secrets may then be referenced by their name in Vault and used to replace values in config such as the following application.yaml example...

datasources:
  default:
    ocid: ocid1.autonomousdatabase.oc1.iad.anuwcl...
    walletPassword: ${ATP_ADMIN_PASSWORD}
    username: micronautdemo
    password: ${ATP_USER_PASSWORD}

This is documented here.

In this way, it is possible to completely decouple the application from environmental configuration and instead access both wallets and passwords from the microservice at runtime.

Conclusion

We have built upon the first blog, which gave examples of how to connect to the Oracle database and containerize for microservice environments, by showing how to do so in a secure, decoupled, and simplified manner.

In the next installment of the series, we will continue to look at various frameworks of these languages and how they provide additional convenience and functionality when creating microservices using the converged Oracle Database.

Please feel free to provide any feedback here, on the workshop, on the GitHub repos, or directly. We are happy to hear from you.