Configuring HashiCorp Vault Custom Policies for API Connect

Note: This article is based on the earlier draft 2026-02-03-hashicorp-vault-custom-policy-apic.md with expanded examples and v12 updates.

Storing secrets — database passwords, API keys, client certificates — directly in your API definitions or configuration files is a security anti-pattern that迟早 leads to credential leakage. HashiCorp Vault is the industry-standard solution for secrets management, and API Connect supports integration via custom GatewayScript policies that retrieve secrets at runtime.

This article covers how to integrate Vault secrets with API Connect, with practical custom policy examples and best practices for vault access.

Why HashiCorp Vault with API Connect?
Architecture Overview
Prerequisites
Configuring Vault Access
Vault Authentication Methods
Custom GatewayScript Policy Examples
Best Practices for Vault Access
Error Handling and Retry Logic

Why HashiCorp Vault with API Connect?

The core benefit is secrets elimination from static configuration. Without Vault:

Database passwords are in config files
API keys are in environment variables or plain text
Certificates are stored on disk with literal filenames
When a credential rotates, you update files and redeploy

With Vault integration:

API Connect retrieves secrets at runtime from Vault
Secrets are never written to disk in your API Connect deployment
When credentials rotate in Vault, API Connect automatically uses the new values
Audit logs in Vault show exactly which workloads accessed which secrets

This is a fundamental shift in security posture, especially important for PCI-DSS, SOC 2, and ISO 27001 compliance programmes.

Architecture Overview

┌──────────────────┐
│   API Consumer   │
└────────┬─────────┘
         │ HTTPS
         ▼
┌──────────────────┐
│  API Gateway     │
│  (GatewayScript) │
│  + vault-utils   │
└────────┬─────────┘
         │ HTTPS (mTLS)
         │ token: s.abc123...
         ▼
┌──────────────────┐
│  HashiCorp Vault │
│  secretsEngine:  │
│  kv-v2 / pki     │
└──────────────────┘

The API Gateway uses GatewayScript with the vault module (or an HTTP-based Vault client) to authenticate to Vault and retrieve secrets at request time.

Prerequisites

API Connect v12 API Gateway (or DataPower Gateway with appropriate firmware)
HashiCorp Vault (v1.12+ recommended) with KV secrets engine v2 enabled
Network connectivity from API Gateway to Vault (port 8200 HTTPS)
Vault token or AppRole credentials for API Connect

Configuring Vault Access

Step 1: Enable the KV Secrets Engine in Vault

# Enable KV v2 secrets engine
vault secrets enable -path=apic-secrets kv-v2

# Verify
vault secrets list

Step 2: Create a Policy for API Connect

Vault policies control what paths API Connect’s token can access:

# apic-policy.hcl
# Policy for API Connect to access secrets at apic-secrets/data/*

path "apic-secrets/data/*" {
  capabilities = ["read", "list"]
}

path "apic-secrets/metadata/*" {
  capabilities = ["list"]
}

# If using dynamic database credentials:
path "database/creds/apic-readonly" {
  capabilities = ["read"]
}

Apply the policy:

vault policy write apic-gateway apic-policy.hcl

Step 3: Create a Token for API Connect

# Create a token with the apic-gateway policy
vault token create \
  -policy=apic-gateway \
  -ttl=24h \
  -display-name="api-connect-gateway"

# Output:
# Token: s.abc123def456...
# Token accessor: xyz789...

Note: For production, prefer AppRole or Kubernetes auth over static tokens. Tokens should have short TTLs and be renewed programmatically.

Vault Authentication Methods

Method 1: Token Authentication (Simplest)

Best for testing. The token is stored in the GatewayScript policy configuration.

var vault = require('vault');
var client = vault.createClient({
  token: 's.abc123def456',
  apiUrl: 'https://vault.example.com:8200'
});

Method 2: AppRole Authentication (Recommended for Production)

AppRole is a machine-to-machine auth method — no username/password involved:

# In Vault: enable AppRole
vault auth enable approle

# Create a role
vault write auth/approle/role/apic-gateway \
  token_ttl=1h \
  token_max_ttl=24h \
  token_policies=apic-gateway

# Get the role_id and secret_id
vault read auth/approle/role/apic-gateway/role-id
vault read -f auth/approle/role/apic-gateway/secret-id

GatewayScript then authenticates using role_id + secret_id to get a short-lived token:

// authenticate with AppRole
var response = https.request({
  hostname: 'vault.example.com',
  port: 8200,
  path: '/v1/auth/approle/login',
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  cert: fs.readFileSync('/path/to/gateway-client.crt'),
  key: fs.readFileSync('/path/to/gateway-client.key')
});

var body = JSON.stringify({
  role_id: 'my-role-id',
  secret_id: 'my-secret-id'
});

var resp = JSON.parse(response.write(body) && response.end() && response.read());
var vaultToken = resp.auth.client_token;

Custom GatewayScript Policy Examples

Example 1: Retrieve a Database Password

This policy retrieves a database password from Vault and sets it as a variable for use in a DataPower DB connection:

// get-db-password.js
var apim = require('apim');
var https = require('https');
var fs = require('fs');

var VAULT_ADDR = 'https://vault.example.com:8200';
var VAULT_TOKEN = apim.getvariable('vault.token'); // Set in policy config
var SECRET_PATH = 'apic-secrets/data/database-payment-svc';

function vaultRequest(method, path, body, token) {
    return new Promise(function(resolve, reject) {
        var options = {
            hostname: VAULT_ADDR.replace('https://', ''),
            port: 443,
            path: path,
            method: method,
            headers: {
                'Content-Type': 'application/json',
                'X-Vault-Token': token || VAULT_TOKEN
            },
            // For production: configure TLS certs for Vault mTLS if needed
            // cert: fs.readFileSync('/path/to/vault-client.crt'),
            // key: fs.readFileSync('/path/to/vault-client.key')
        };

        var req = https.request(options, function(res) {
            var data = '';
            res.on('data', function(chunk) { data += chunk; });
            res.on('end', function() {
                try {
                    resolve(JSON.parse(data));
                } catch(e) {
                    reject(new Error('Failed to parse Vault response: ' + data));
                }
            });
        });

        req.on('error', reject);
        if (body) req.write(JSON.stringify(body));
        req.end();
    });
}

async function getSecret(path) {
    try {
        var result = await vaultRequest('GET', '/v1/' + path + '?version=latest');
        if (!result.data || !result.data.data) {
            throw new Error('Secret not found: ' + path);
        }
        return result.data.data;
    } catch (e) {
        apim.error('vault_error', 500, 'Failed to retrieve secret from Vault: ' + e.message);
        throw e;
    }
}

async function main() {
    try {
        var secret = await getSecret(SECRET_PATH);
        // Set the password as a variable for use in downstream policies
        apim.setvariable('private.db_password', secret.password);
        apim.setvariable('private.db_username', secret.username);
        apim.setvariable('private.db_host', secret.host);
        
        apim.console.log('Vault: Retrieved DB credentials for ' + secret.db_name);
    } catch (e) {
        apim.console.error('Vault: Failed to get secret - ' + e.message);
        // Don't re-throw — apim.error() already called in getSecret
    }
}

main();

Example 2: Retrieve an OAuth Client Secret

Useful when your API acts as an OAuth client to a backend service:

// get-oauth-client-secret.js
var apim = require('apim');
var https = require('https');

var VAULT_ADDR = 'https://vault.example.com:8200';
var SECRET_PATH = 'apic-secrets/data/oauth-payment-partner';

async function getSecret(path) {
    // Uses the same vaultRequest helper described above
    var vaultRequest = require('./vault-request-helper');
    var result = await vaultRequest.get(path);
    return result.data.data;
}

async function main() {
    try {
        var secret = await getSecret(SECRET_PATH);
        
        // Set as variables for use in the OAuth assembly policy
        apim.setvariable('oauth.client_id', secret.client_id);
        apim.setvariable('oauth.client_secret', secret.client_secret);
        apim.setvariable('oauth.token_url', secret.token_url);
        
        apim.console.log('Vault: Retrieved OAuth client credentials');
    } catch (e) {
        apim.error('vault_error', 500, 'Failed to get OAuth credentials: ' + e.message);
    }
}

main();

Example 3: Certificate Retrieval via Vault PKI

If you’re using Vault’s PKI engine to generate dynamic certificates:

// get-dynamic-cert.js
var apim = require('apim');
var https = require('https');

var VAULT_ADDR = 'https://vault.example.com:8200';
var ROLE_NAME = 'apic-backend-client';

async function generateCertificate() {
    var vaultRequest = require('./vault-request-helper');
    
    var result = await vaultRequest.post('/v1/pki/issue/' + ROLE_NAME, {
        common_name: 'backend-payment-svc.example.com',
        ttl: '1h'
    });
    
    return result.data;
}

async function main() {
    try {
        var certBundle = await generateCertificate();
        
        // Store the certificate and key in variables
        apim.setvariable('private.backend_cert', certBundle.certificate);
        apim.setvariable('private.backend_private_key', certBundle.private_key);
        apim.setvariable('private.backend_ca_cert', certBundle.issuing_ca_certificate);
        
        // Set TTL as a variable so we know when to refresh
        apim.setvariable('private.cert_expires_at', certBundle.expiration);
        
        apim.console.log('Vault: Generated dynamic certificate, expires at ' + certBundle.expiration);
    } catch (e) {
        apim.error('vault_error', 500, 'Failed to generate certificate: ' + e.message);
    }
}

main();

Note: Dynamic certificates from Vault PKI have short TTLs (e.g., 1 hour). You should implement caching and only request new certificates when the current one is close to expiry, to avoid excessive Vault calls.

Best Practices for Vault Access

1. Use AppRole, Not Static Tokens

Static tokens are a security risk. If a token is compromised, it has indefinite access until revoked. AppRole provides short-lived credentials that are exchanged for short-lived Vault tokens.

2. Implement Secret Caching

Calling Vault on every request adds latency and load. Cache secrets in memory (or a short-lived local cache) and only refresh when approaching TTL expiry:

var cache = {
    secret: null,
    expiresAt: 0
};

async function getCachedSecret(path) {
    var now = Date.now();
    if (cache.secret && cache.expiresAt > now + 60000) { // Refresh 1 min before expiry
        return cache.secret;
    }
    
    var secret = await getSecret(path);
    cache.secret = secret;
    cache.expiresAt = (secret._ttl_seconds || 3600) * 1000 + now;
    return secret;
}

3. Use mTLS for Vault Communication

Configure mutual TLS between API Gateway and Vault so both sides verify certificates:

var options = {
    hostname: 'vault.example.com',
    port: 443,
    path: '/v1/...',
    cert: fs.readFileSync('/path/to/gateway-client.crt'),
    key: fs.readFileSync('/path/to/gateway-client.key'),
    ca: fs.readFileSync('/path/to/vault-ca.crt')
};

4. Log Carefully

Vault audit logs every request by default. Don’t log secret values — only log the fact that a secret was accessed:

// Good:
apim.console.log('Vault: Retrieved secret from apic-secrets/data/payment-db');

// Bad:
apim.console.log('Vault: Retrieved secret password=' + secret.password); // Never do this!

5. Handle Vault Unavailability Gracefully

If Vault is unreachable, your API should fail in a predictable way rather than returning potentially incorrect data:

async function main() {
    try {
        // ... retrieve secret
    } catch (e) {
        // Vault is unavailable — fail the request rather than proceeding
        apim.error('vault_unavailable', 503, 'Secret service temporarily unavailable');
    }
}

Error Handling and Retry Logic

async function vaultRequestWithRetry(method, path, body, token, retries) {
    retries = retries || 3;
    var attempt = 0;
    
    while (attempt < retries) {
        try {
            var result = await vaultRequest(method, path, body, token);
            return result;
        } catch (e) {
            attempt++;
            if (attempt >= retries) throw e;
            var delay = Math.min(1000 * Math.pow(2, attempt), 10000);
            apim.console.log('Vault request failed, retrying in ' + delay + 'ms (attempt ' + attempt + '/' + retries + ')');
            await new Promise(function(resolve) { setTimeout(resolve, delay); });
        }
    }
}

Summary

Integrating HashiCorp Vault with API Connect via custom GatewayScript policies is a powerful pattern for eliminating secrets from your API configuration. By retrieving secrets at runtime, you gain a single source of truth for credentials, automatic rotation support, and a comprehensive audit trail.

The key to a production-ready implementation is AppRole authentication (not static tokens), secret caching to avoid Vault on every request, proper error handling for Vault unavailability, and careful logging that never exposes secret values.

Screenshots to Capture

[Placeholder: /images/vault-policy-creation-ui.png] What to capture: The HashiCorp Vault UI showing the creation of an AppRole policy for API Connect, with the policy editor displaying path permissions for secrets access. Where: HashiCorp Vault → Policies → Create policy → AppRole policy editor
[Placeholder: /images/vault-approle-role-config.png] What to capture: The Vault AppRole role configuration showing token TTL settings, bound secret ID, and the associated policy assignment. Where: HashiCorp Vault → Auth → AppRole → [apic-gateway role] → View role configuration
[Placeholder: /images/vault-secret-retrieval-log.png] What to capture: A DataPower/API Gateway log excerpt showing a successful Vault secret retrieval, including the secret path and the absence of any secret values in the log (confirming proper logging hygiene). Where: DataPower Gateway logs or API Gateway logs (filtered for Vault access entries)

Questions about Vault integration with API Connect? Find me on Twitter @cminion.

Chris Phillips’ Blog - API, Integration and Governance

Home

About

Search

SEO Improvements for Recent Posts (Last 6 Months)

Jekyll Upgrade Notes

Categories

Guest Authors