MEMO-041: Authentication and Authorization Integration Testing Guide

Overview

This memo documents the comprehensive end-to-end authentication and authorization integration testing infrastructure for Prism, including JWT validation, namespace-based authorization, and auth context pass-through to pattern runners.

Implemented in PR #33: Complete E2E JWT Authentication and Authorization Integration

Background

The authentication and authorization integration enables:

• JWT token validation with Dex OIDC provider
• Namespace-based read/write permissions
• Auth context pass-through to pattern runners via HTTP/2 headers
• Zero-boilerplate SDK for pattern runners to extract auth context
• Distributed tracing with unique request trace IDs
• Audit logging with user identity at pattern runner level

Test Architecture

The auth integration test (tests/testing/auth_integration_test.go) provides end-to-end testing using:

• Dex - OIDC provider for JWT token issuance
• prism-proxy - Rust proxy with JWT validation and authorization
• keyvalue-runner - MemStore-backed KeyValue service with auth SDK
• Test client - Embedded Go client that acquires tokens and makes requests

Architecture Diagram

┌─────────────┐
│ Test Client │ (Go test code - acquires JWT, makes requests)
└──────┬──────┘
       │
       │ Bearer token (JWT)
       ▼
┌─────────────┐        ┌─────────────┐
│    Dex      │◀───────│ Prism Proxy │
│   (OIDC)    │ JWKS   │  (Rust)     │
└─────────────┘        └──────┬──────┘
                              │
                              │ Auth context headers
                              │ (trace-id, user-id, permission)
                              ▼
                       ┌──────────────┐
                       │  keyvalue-   │
                       │   runner     │
                       │  (MemStore)  │
                       └──────────────┘

Implementation Status

What Works Now ✅

• JWT validation with Dex OIDC provider
• Namespace-based authorization policies
• Auth context pass-through via HTTP/2 HEADERS
• Pattern runner SDK for auth context extraction
• 10 auth test scenarios (token acquisition, permissions, denials)
• Dex testcontainer backend infrastructure
• Zero-copy TCP forwarding with selective HEADERS modification
• Distributed tracing with UUID v4 trace IDs
• Automatic audit logging via gRPC interceptors

Components Implemented

1. Proxy Authentication (prism-proxy/src/main.rs):

   • AuthContext struct for user identity and permissions
   • determine_permission() - Maps gRPC methods to Read/Write
   • handle_connection() - JWT validation and authorization
   • inject_context_headers() - Modifies HTTP/2 HEADERS frames
   • Configuration: PRISM_AUTH_ENABLED, DEX_ISSUER, DEX_CLIENT_ID

2. Authorization Policies (prism-proxy/src/auth.rs):

   • team-alpha: dev (read+write), alice (read only)
   • team-beta: admin (read+write), alice (read+write), bob (read only)
   • shared: everyone (read+write)

3. Pattern Runner SDK (pkg/plugin/):

   • auth_context.go - AuthContext type and extraction helpers
   • auth_interceptor.go - Automatic logging interceptors
   • Zero-boilerplate: ExtractAuthContext(ctx)

4. Test Infrastructure (tests/testing/):

   • backends/dex.go - Dex testcontainer backend
   • auth_integration_test.go - 10 test scenarios

Running the Tests

Test Command

cd tests/testing
go test -v -run TestAuthIntegration

Test Scenarios Covered

The integration test validates:

1. JWT Token Acquisition - Acquire tokens from Dex for test users
2. Read Permissions - Verify read-only access to namespaces
3. Write Permissions - Verify read+write access to namespaces
4. Permission Denied (403) - Verify permission denial scenarios
5. Unauthenticated Requests (401) - Verify auth requirement enforcement
6. Namespace Isolation - Verify users can't access unauthorized namespaces
7. Method-Based Permissions - Get→Read, Set→Write
8. Auth Context Pass-Through - Verify headers reach pattern runners
9. Trace ID Generation - Verify unique trace IDs per request
10. Audit Logging - Verify user identity in pattern runner logs

Test Users

Test users are configured in local-dev/dex/config.yaml:

Email	Password	User ID
dev@local.prism	password	08a8684b-db88-4b73-90a9-3cd1661f5466
admin@local.prism	password	3b241101-e2bb-4255-8caf-4136c566a962
alice@example.com	password	41331323-6f44-45e6-b3b9-2c4b60c02be5
bob@example.com	password	7e8b6c3e-5e37-4f5e-9e0e-3e5f8f8f8f8f

Auth Context Pass-Through

The proxy injects authentication context headers into every request forwarded to pattern runners. This enables:

• Audit Logging - Log requests with user identity
• Secondary Authorization - Pattern runners can add additional checks
• Distributed Tracing - Correlate requests across services using trace IDs
• User-Specific Behavior - Rate limiting, quotas, per-user metrics

Context Headers

Headers injected by proxy (per-call):

• x-prism-trace-id - Unique UUID for request tracing
• x-prism-user-id - User ID from JWT subject claim
• x-prism-user-email - User email (if present in JWT)
• x-prism-namespace - Namespace being accessed
• x-prism-permission - Permission level granted (read or write)
• x-prism-scopes - OAuth scopes (comma-separated, if available)

Pattern runners receive these headers on every gRPC call and can extract them from the gRPC metadata.

Technical Details

• Per-Call Injection - Stateless operation, unique trace IDs per request
• Zero-Copy Forwarding - Only HEADERS frames modified via HPACK encoding
• HTTP/2 Selective Modification - Data frames pass through untouched
• No Proxy State - Scales horizontally without session management

Using Auth Context in Pattern Runners

The pkg/plugin SDK provides built-in support for extracting and using auth context. Pattern runners get automatic access to user identity, permissions, and tracing information.

Step 1: Add Interceptor to gRPC Server

import "github.com/jrepp/prism-data-layer/pkg/plugin"

grpcServer := grpc.NewServer(
    grpc.UnaryInterceptor(plugin.AuthLoggingInterceptor()),
    grpc.StreamInterceptor(plugin.AuthStreamInterceptor()),
)

The interceptor automatically logs all requests with auth context.

Step 2: Extract Context in Handlers

func (s *MyService) Set(ctx context.Context, req *pb.SetRequest) (*pb.SetResponse, error) {
    // Extract auth context from request
    authCtx := plugin.ExtractAuthContext(ctx)

    // Log with user identity and trace ID
    slog.Info("Set operation",
        "key", req.Key,
        "trace_id", authCtx.TraceID,
        "user_id", authCtx.UserID,
        "namespace", authCtx.Namespace,
        "permission", authCtx.Permission,
    )

    // Check permissions (optional - proxy already enforced this)
    if !authCtx.HasPermission(plugin.PermissionWrite) {
        return &pb.SetResponse{
            Success: false,
            Error:   "insufficient permissions",
        }, nil
    }

    // Your handler logic here...
}

Step 3: Use Helper Methods

authCtx := plugin.ExtractAuthContext(ctx)

// Check authentication status
if authCtx.IsAuthenticated {
    // User is authenticated
}

// Check permissions
if authCtx.HasPermission(plugin.PermissionRead) {
    // User has read permission
}

if authCtx.HasPermission(plugin.PermissionWrite) {
    // User has write permission (includes read)
}

// Check OAuth scopes
if authCtx.HasScope("admin") {
    // User has admin scope
}

// Get structured logging fields
slog.Info("operation", authCtx.LogFields()...)

AuthContext API

type AuthContext struct {
    TraceID         string   // Unique trace ID
    UserID          string   // User ID (JWT subject)
    UserEmail       string   // User email
    Namespace       string   // Namespace
    Permission      string   // "read" or "write"
    Scopes          []string // OAuth scopes
    IsAuthenticated bool     // Auth status
}

Benefits for Pattern Runners

• ✅ Automatic audit logging with user identity
• ✅ Distributed tracing with unique trace IDs
• ✅ Secondary authorization checks (if needed)
• ✅ User-specific behavior (rate limiting, quotas)
• ✅ Zero-boilerplate - just call ExtractAuthContext(ctx)

Configuration

Proxy Configuration (Environment Variables)

# Enable authentication
export PRISM_AUTH_ENABLED=true

# Dex OIDC configuration
export DEX_ISSUER=http://localhost:5556/dex
export DEX_CLIENT_ID=prismctl

Authorization Policies

Authorization policies are configured in prism-proxy/src/auth.rs:

pub fn with_test_policies() -> Self {
    let mut policies = HashMap::new();

    // User IDs from Dex config
    const DEV_USER_ID: &str = "08a8684b-db88-4b73-90a9-3cd1661f5466";
    const ADMIN_USER_ID: &str = "3b241101-e2bb-4255-8caf-4136c566a962";
    const ALICE_USER_ID: &str = "41331323-6f44-45e6-b3b9-2c4b60c02be5";
    const BOB_USER_ID: &str = "7e8b6c3e-5e37-4f5e-9e0e-3e5f8f8f8f8f";

    // team-alpha: dev has read+write, alice has read only
    policies.insert("team-alpha".to_string(), NamespacePolicy {
        read_users: vec![DEV_USER_ID.to_string(), ALICE_USER_ID.to_string()],
        write_users: vec![DEV_USER_ID.to_string()],
    });

    // team-beta: admin has read+write, alice has read+write, bob has read only
    policies.insert("team-beta".to_string(), NamespacePolicy {
        read_users: vec![ADMIN_USER_ID.to_string(), ALICE_USER_ID.to_string(), BOB_USER_ID.to_string()],
        write_users: vec![ADMIN_USER_ID.to_string(), ALICE_USER_ID.to_string()],
    });

    // shared: everyone has read+write
    policies.insert("shared".to_string(), NamespacePolicy {
        read_users: vec![DEV_USER_ID.to_string(), ADMIN_USER_ID.to_string(), ALICE_USER_ID.to_string(), BOB_USER_ID.to_string()],
        write_users: vec![DEV_USER_ID.to_string(), ADMIN_USER_ID.to_string(), ALICE_USER_ID.to_string(), BOB_USER_ID.to_string()],
    });

    Self { policies }
}

Comparison to Old E2E Test

Old Approach (tests/e2e/auth-integration/)

• ❌ Docker Compose orchestration (separate process, harder to debug)
• ❌ Custom Dockerfiles for each component
• ❌ Separate client binary
• ❌ Manual service coordination
• ✅ Full auth/authz testing

New Approach (tests/testing/)

• ✅ Testcontainers (runs in Go test process, easier to debug)
• ✅ Uses existing binaries (no custom Dockerfiles)
• ✅ Client logic embedded in test
• ✅ Automatic cleanup
• ✅ Consistent with other integration tests (redis, nats, postgres)
• ✅ Complete auth integration with SDK and context pass-through

Related Files

Proxy Implementation

• prism-proxy/src/main.rs - Auth integration, context injection, permission determination
• prism-proxy/src/config/mod.rs - Auth configuration support
• prism-proxy/src/auth.rs - JWT validation and authorization policies
• prism-proxy/src/lib.rs - Exported types and configuration
• prism-proxy/Cargo.toml - Dependencies (uuid for trace IDs)

Pattern Runner SDK

• pkg/plugin/auth_context.go - AuthContext type and extraction helpers
• pkg/plugin/auth_interceptor.go - gRPC interceptors for automatic logging
• patterns/keyvalue/grpc_server.go - Example SDK integration

Test Infrastructure

• tests/testing/backends/dex.go - Dex testcontainer backend
• tests/testing/auth_integration_test.go - Main integration test
• local-dev/dex/config.yaml - Dex OIDC configuration

Key Innovations

1. Zero-Copy Forwarding Preserved

TCP-level transparent proxy maintains zero-copy forwarding while extracting metadata from HTTP/2 HEADERS frames. Only HEADERS frames are modified via HPACK encoding - data frames pass through untouched.

2. Per-Call Auth Context Injection

Stateless operation with unique trace IDs per request enables:

• Distributed tracing across proxy and pattern runners
• User-specific audit logging at pattern runner level
• No session state in proxy (scales horizontally)

3. Zero-Boilerplate SDK

Pattern runners just call ExtractAuthContext(ctx) to get full auth context with user identity, permissions, and trace ID. No manual header extraction or parsing required.

4. Defense-in-Depth Security

Both proxy AND pattern runners can validate authorization:

• Proxy enforces namespace-based policies
• Pattern runners can add secondary checks (e.g., row-level security)
• Zero-trust architecture - pattern runners validate auth independently

Security Considerations

JWT Validation

• Tokens validated using JWKS from Dex OIDC provider
• Issuer and audience claims verified
• Token expiration enforced
• Signature verification using public keys from JWKS endpoint

Authorization Enforcement

• Namespace-based policies enforced at proxy level
• Method-based permissions (Get→Read, Set→Write)
• Pattern runners can add additional authorization checks
• User identity propagated to pattern runners for audit logging

Trace ID Security

• UUID v4 trace IDs are non-guessable
• Generated per-request for correlation, not authentication
• Safe to include in logs and distributed tracing systems

References

• ADR-007: Authentication and Authorization
• RFC-011: Data Proxy Authentication
• RFC-019: Plugin SDK Authorization Layer
• PR #33: Complete E2E JWT Authentication and Authorization Integration
• Dex Documentation
• testcontainers-go

Next Steps

1. Test Module Dependencies - Resolve Go module dependency issues (separate infrastructure task)
2. Additional Pattern Runners - Add auth integration tests for consumer, producer patterns
3. OAuth2 Device Code Flow - Add device code flow for CLI authentication
4. mTLS Support - Add mutual TLS for service-to-service authentication
5. Policy as Code - Externalize authorization policies to config files or policy engine (Topaz)
6. Token Refresh - Add automatic token refresh for long-running clients
7. Scope-Based Authorization - Add OAuth scope validation in addition to namespace policies