MEMO-091: Local Vault + Dex Setup Guide

Executive Summary

This guide provides step-by-step instructions for setting up a local authentication testing environment with HashiCorp Vault and Dex OIDC provider. This environment enables developers to test the complete authentication flow (JWT validation, Vault token exchange, dynamic credentials) without requiring production infrastructure.

What You'll Set Up:

• HashiCorp Vault (dev mode) with JWT auth method
• Dex OIDC provider with test users
• Redis backend with Vault-managed credentials
• Example pattern runners configured for authentication

Time to Complete: ~30 minutes

Prerequisites

Required:

• Docker Desktop or Podman installed
• docker-compose or podman-compose
• vault CLI tool (installation)
• curl and jq for testing

Optional:

• prismctl CLI with auth support
• Pattern runners built locally

Architecture Overview

┌────────────────────────────────────────────────────────────────┐
│                  Local Authentication Stack                    │
│                                                                │
│  ┌──────────────┐      ┌──────────────┐      ┌─────────────┐ │
│  │     Dex      │      │    Vault     │      │   Redis     │ │
│  │              │      │              │      │             │ │
│  │  OIDC        │◄─────┤  JWT Auth    │◄─────┤  Dynamic    │ │
│  │  Provider    │      │  Method      │      │  Credentials│ │
│  │              │      │              │      │             │ │
│  │  Port: 5556  │      │  Port: 8200  │      │  Port: 6379 │ │
│  └───────┬──────┘      └───────┬──────┘      └──────┬──────┘ │
│          │                     │                    │        │
│          │ 1. Get JWT          │ 2. Exchange        │ 3. Use │
│          │                     │    for Vault token │   Creds│
│          ▼                     ▼                    ▼        │
│  ┌────────────────────────────────────────────────────────┐  │
│  │            Pattern Runner (KeyValue/Consumer/Producer) │  │
│  └────────────────────────────────────────────────────────┘  │
└────────────────────────────────────────────────────────────────┘

Step 1: Create Docker Compose Configuration

Create docker-compose-auth.yml in your project root:

version: '3.8'

services:
  # Dex OIDC Provider
  dex:
    image: ghcr.io/dexidp/dex:v2.37.0
    ports:
      - "5556:5556"
    volumes:
      - ./local-dev/dex-config.yaml:/etc/dex/config.yaml:ro
    command: ["dex", "serve", "/etc/dex/config.yaml"]
    networks:
      - prism-local

  # HashiCorp Vault (Dev Mode)
  vault:
    image: hashicorp/vault:1.15
    ports:
      - "8200:8200"
    environment:
      VAULT_DEV_ROOT_TOKEN_ID: "root"
      VAULT_DEV_LISTEN_ADDRESS: "0.0.0.0:8200"
      VAULT_LOG_LEVEL: "info"
    cap_add:
      - IPC_LOCK
    networks:
      - prism-local

  # Redis (Backend for testing)
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    command: ["redis-server", "--requirepass", "redis-admin-password"]
    networks:
      - prism-local

  # NATS (Backend for testing)
  nats:
    image: nats:2.10-alpine
    ports:
      - "4222:4222"
      - "8222:8222"  # HTTP monitoring
    networks:
      - prism-local

networks:
  prism-local:
    driver: bridge

Step 2: Configure Dex OIDC Provider

Create local-dev/dex-config.yaml:

issuer: http://localhost:5556/dex

storage:
  type: memory

web:
  http: 0.0.0.0:5556

# Static clients (applications that will use Dex)
staticClients:
  - id: prism-patterns
    redirectURIs:
      - 'http://localhost:8080/callback'
      - 'http://localhost:3000/callback'
    name: 'Prism Patterns'
    secret: prism-test-secret

  - id: prismctl
    redirectURIs:
      - 'http://localhost:8000/callback'
      - 'urn:ietf:wg:oauth:2.0:oob'  # For CLI device flow
    name: 'Prism CLI'
    secret: prismctl-secret
    public: true  # CLI is a public client

# OAuth2 settings
oauth2:
  skipApprovalScreen: true  # Auto-approve for local testing

# Connector: Static users for testing
connectors:
  - type: mockCallback
    id: mock
    name: Mock

# Static passwords (test users)
enablePasswordDB: true
staticPasswords:
  - email: "alice@example.com"
    hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"  # Password: password
    username: "alice"
    userID: "08a8684b-db88-4b73-90a9-3cd1661f5466"
    groups:
      - team-alpha
      - developers

  - email: "bob@example.com"
    hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"  # Password: password
    username: "bob"
    userID: "41331323-6f44-45e6-b3b9-2c4b2c11a738"
    groups:
      - team-beta
      - developers

  - email: "service@example.com"
    hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"  # Password: password
    username: "service-account"
    userID: "service-123"
    groups:
      - services

Step 3: Start the Stack

# Create local-dev directory
mkdir -p local-dev

# Start all services
docker-compose -f docker-compose-auth.yml up -d

# Wait for services to be ready (30 seconds)
sleep 30

# Verify Dex is running
curl -s http://localhost:5556/dex/.well-known/openid-configuration | jq .

# Verify Vault is running
export VAULT_ADDR='http://localhost:8200'
export VAULT_TOKEN='root'
vault status

Expected Output:

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    1
Threshold       1
Version         1.15.0
Storage Type    inmem
Cluster Name    vault-cluster-xxxxx
Cluster ID      xxxxx
HA Enabled      false

Step 4: Configure Vault JWT Auth Method

Create local-dev/setup-vault.sh:

#!/bin/bash

set -e

export VAULT_ADDR='http://localhost:8200'
export VAULT_TOKEN='root'

echo "==> Configuring Vault JWT Auth Method"

# Enable JWT auth method
vault auth enable jwt

# Configure JWT auth to use Dex as OIDC provider
vault write auth/jwt/config \
    oidc_discovery_url="http://localhost:5556/dex" \
    oidc_client_id="prism-patterns" \
    oidc_client_secret="prism-test-secret" \
    default_role="prism-patterns"

# Create role for pattern plugins
vault write auth/jwt/role/prism-patterns \
    role_type="jwt" \
    bound_audiences="prism-patterns" \
    user_claim="email" \
    groups_claim="groups" \
    token_ttl="1h" \
    token_max_ttl="2h" \
    token_policies="prism-patterns-policy"

echo "==> Configuring Vault Database Secrets Engine"

# Enable database secrets engine
vault secrets enable database

# Configure Redis connection
vault write database/config/redis \
    plugin_name="redis-database-plugin" \
    host="redis" \
    port=6379 \
    username="default" \
    password="redis-admin-password" \
    allowed_roles="redis-role"

# Create role for dynamic Redis credentials
vault write database/roles/redis-role \
    db_name="redis" \
    creation_statements='["ACL SETUSER {{username}} on >{{password}} ~* +@all"]' \
    revocation_statements='["ACL DELUSER {{username}}"]' \
    default_ttl="1h" \
    max_ttl="2h"

echo "==> Creating Vault Policy"

# Create policy file
cat > /tmp/prism-patterns-policy.hcl <<EOF
# Allow token renewal
path "auth/token/renew-self" {
  capabilities = ["update"]
}

# Allow lease renewal
path "sys/leases/renew" {
  capabilities = ["update"]
}

# Allow lease revocation
path "sys/leases/revoke" {
  capabilities = ["update"]
}

# Allow reading Redis credentials
path "database/creds/redis-role" {
  capabilities = ["read"]
}
EOF

# Write policy to Vault
vault policy write prism-patterns-policy /tmp/prism-patterns-policy.hcl

echo "✅ Vault configuration complete!"

Make executable and run:

chmod +x local-dev/setup-vault.sh
./local-dev/setup-vault.sh

Step 5: Test Authentication Flow

5.1: Get JWT Token from Dex

# Use password grant (testing only - not for production!)
curl -X POST http://localhost:5556/dex/token \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=password" \
  -d "username=alice@example.com" \
  -d "password=password" \
  -d "client_id=prism-patterns" \
  -d "client_secret=prism-test-secret" \
  -d "scope=openid email profile groups" | jq -r '.id_token' > /tmp/jwt_token.txt

# View token
cat /tmp/jwt_token.txt

5.2: Exchange JWT for Vault Token

export JWT_TOKEN=$(cat /tmp/jwt_token.txt)

# Authenticate with Vault using JWT
vault write -field=token auth/jwt/login \
  role=prism-patterns \
  jwt=$JWT_TOKEN > /tmp/vault_token.txt

# Use Vault token
export VAULT_TOKEN=$(cat /tmp/vault_token.txt)

echo "Vault token: $VAULT_TOKEN"

5.3: Fetch Dynamic Redis Credentials

# Get Redis credentials from Vault
vault read database/creds/redis-role

# Output:
# Key                Value
# ---                -----
# lease_id           database/creds/redis-role/xxxxx
# lease_duration     1h
# lease_renewable    true
# password           A1a-xxxxxxxxx
# username           v-jwt-alice-xxxxx

5.4: Test Redis Connection with Dynamic Credentials

# Extract credentials
export REDIS_USER=$(vault read -field=username database/creds/redis-role)
export REDIS_PASS=$(vault read -field=password database/creds/redis-role)

# Test connection
redis-cli -h localhost -p 6379 --user $REDIS_USER --pass $REDIS_PASS PING

# Output: PONG

Step 6: Configure Pattern Runner with Authentication

Create examples/configs/keyvalue-local-auth.yaml:

namespaces:
  - name: local-test
    pattern: keyvalue
    pattern_version: 0.1.0
    description: Local testing with authentication enabled
    backend: redis
    backend_config:
      host: localhost
      port: 6379
      # Credentials will be injected from Vault session

    # Authentication configuration
    auth:
      enabled: true

      jwt:
        issuer: http://localhost:5556/dex
        audience: prism-patterns
        skip_expiry_check: false
        skip_issuer_check: false

      vault:
        address: http://localhost:8200
        role: prism-patterns
        auth_path: auth/jwt
        secret_path: database/creds/redis-role
        tls:
          enabled: false  # Local dev uses HTTP
          skip_verify: true

      session_ttl: 1h
      idle_timeout: 30m

Step 7: Run Pattern Runner with Authentication

# Set JWT token environment variable
export PRISM_JWT_TOKEN=$(cat /tmp/jwt_token.txt)

# Run KeyValue runner
./patterns/keyvalue/cmd/keyvalue-runner/keyvalue-runner \
  -config examples/configs/keyvalue-local-auth.yaml \
  -grpc-port 8081

# Output:
# [KEYVALUE-RUNNER] Authentication enabled, initializing Vault integration...
# [KEYVALUE-RUNNER] Creating token validator (issuer: http://localhost:5556/dex, audience: prism-patterns)
# [KEYVALUE-RUNNER] ✅ Token validator created
# [KEYVALUE-RUNNER] Creating Vault client (address: http://localhost:8200, role: prism-patterns)
# [KEYVALUE-RUNNER] ✅ Vault client created
# [KEYVALUE-RUNNER] Creating session manager...
# [KEYVALUE-RUNNER] ✅ Session manager created (session_ttl: 1h0m0s, idle_timeout: 30m0s)
# [KEYVALUE-RUNNER] ✅ Session middleware created with Vault integration enabled

Step 8: Test End-to-End Flow

8.1: Make Authenticated Request

# Use grpcurl with JWT token
grpcurl -plaintext \
  -H "Authorization: Bearer $(cat /tmp/jwt_token.txt)" \
  -d '{"key": "test-key", "value": "dGVzdC12YWx1ZQ=="}' \
  localhost:8081 \
  prism.patterns.keyvalue.v1.KeyValueService/Set

# Output:
# {
#   "success": true
# }

8.2: Verify Session Created

Check pattern runner logs for session creation:

[SESSION-MIDDLEWARE] Creating new session...
[SESSION-MIDDLEWARE] Created new session: abc-123 (user: alice@example.com)
[SESSION-MIDDLEWARE] Injected credentials: username=v-jwt-alice-xyz, lease=vault-lease-456

8.3: Verify Dynamic Credentials

# List active Redis users (should see dynamic user)
redis-cli -h localhost -p 6379 -a redis-admin-password ACL LIST

# Output includes:
# user v-jwt-alice-xyz on ~* +@all

Troubleshooting

Issue: "authentication failed: no JWT token found"

Cause: JWT token not provided in request

Solution: Add JWT token to Authorization header:

-H "Authorization: Bearer <your-jwt-token>"

Issue: "vault authentication failed"

Cause: Vault JWT auth not configured or Dex unreachable

Solution: Verify Vault can reach Dex:

# From inside Vault container
docker exec -it <vault-container-id> sh
apk add curl
curl http://dex:5556/dex/.well-known/openid-configuration

Issue: "failed to fetch credentials from Vault"

Cause: Redis plugin not loaded or policy insufficient

Solution: Verify Redis plugin and policy:

vault read database/config/redis
vault policy read prism-patterns-policy

Issue: "Redis connection refused"

Cause: Pattern runner can't reach Redis

Solution: Use host.docker.internal if runner is in Docker:

backend_config:
  host: host.docker.internal  # Not localhost
  port: 6379

Cleanup

# Stop all services
docker-compose -f docker-compose-auth.yml down -v

# Remove generated tokens
rm -f /tmp/jwt_token.txt /tmp/vault_token.txt

Next Steps

1. Test with Consumer/Producer: Follow same pattern with other runners
2. Add More Users: Update dex-config.yaml with additional test users
3. Test Credential Renewal: Let session run for 30 minutes and verify renewal
4. Test Session Expiry: Wait for session TTL and verify new credentials fetched
5. Performance Testing: Measure session creation latency with multiple concurrent requests

Related Documents

• RFC-062: Unified Authentication and Session Management
• MEMO-083: Phase 2 Vault Integration Implementation Plan
• MEMO-089: Session Middleware Integration Guide
• MEMO-092: Vault Operator Guide (Pending)
• MEMO-093: Vault Troubleshooting Guide (Pending)

Revision History

• 2025-11-18: Initial local Vault + Dex setup guide for authentication testing