configurationarchitecturenamespacesbackendsauthorizationpatternsunification

Status: ProposedAuthor: Platform TeamCreated: Nov 14, 2025Updated: Nov 14, 2025

Unified Configuration Model

Abstract

This RFC proposes a unified configuration model for Prism, resolving terminology inconsistencies and clarifying the hierarchy of configuration decisions. It establishes a clear framework for how configuration flows from application owner intent through platform policy to runtime execution.

Motivation

Current State: Configuration Across Multiple Documents

Prism's configuration model is currently documented across multiple RFCs and ADRs:

ADR-002: Client-Originated Configuration (protobuf-based needs)
ADR-006: Namespace and Multi-Tenancy (isolation, sharding)
ADR-007: Authentication and Authorization (mTLS, namespace RBAC)
ADR-022: Dynamic Client Configuration (runtime pattern config)
ADR-050: Topaz Policy Authorization (fine-grained RBAC/ABAC)
RFC-014: Layered Data Access Patterns (3-layer architecture)
RFC-027: Namespace Configuration Client Perspective (user-facing config)
RFC-039: Backend Configuration Registry (backend/frontend registries)

Problems

Terminology Inconsistencies
- "api" vs "pattern" vs "client_api" used interchangeably
- "requirements" vs "needs" vs "guarantees"
- "capabilities" vs "interfaces" for backend features
- "permissions" vs "authorization" vs "access control"
Authorization Boundary Confusion
- ADR-002 defines 3 permission levels (Guided, Advanced, Expert)
- ADR-050 introduces Topaz with different RBAC model
- RFC-027 has team quotas separate from permission levels
- Unclear how these relate to each other
Configuration Hierarchy Unclear
- What overrides what: namespace config vs global config?
- Frontend bindings (RFC-039) vs client API selection (RFC-027) relationship?
- Pattern config vs backend slot config separation?
Multi-Tenancy Model Gaps
- Team vs namespace vs tenant relationship unclear
- Quota enforcement at which level?
- Cross-namespace resource sharing rules?
Developer/Operator/User Responsibilities
- Which persona controls which configuration?
- How do configurations evolve (dev → staging → prod)?
- What can be changed without breaking clients?

Goals

Unified Terminology: Single consistent vocabulary across all configuration documents
Clear Hierarchy: Explicit precedence rules for configuration layers
Role Clarity: Define exactly what each persona (User, Developer, Operator) controls
Authorization Model: Consolidate permission levels, RBAC, and policy enforcement
Mental Model: Provide clear conceptual framework for understanding configuration flow

Non-Goals

Implementation Changes: This RFC consolidates existing designs, doesn't change implementations
New Features: No new configuration capabilities, only clarification
Migration Plan: How to migrate existing configs is out of scope
Tooling Changes: Admin CLI/UI changes are separate work

Unified Configuration Model

Core Principles

Declarative Over Imperative: Users declare what they need, not how to implement it
Separation of Concerns: Clear boundaries between user, developer, operator responsibilities
Progressive Disclosure: Simple defaults with escape hatches for advanced use cases
Policy-Driven: Authorization boundaries enforced through centralized policies
Immutable Contracts: Client-facing APIs remain stable, internal implementation can evolve

Configuration Layers (Top to Bottom)

┌─────────────────────────────────────────────────────────────┐
│ Layer 1: User Request (Application Owner)                   │
│ - Namespace name                                            │
│ - Access pattern intent (queue, pubsub, keyvalue, etc.)    │
│ - Functional requirements ("needs")                         │
│ - Access control (team ownership, consumer services)        │
│ - Compliance policies (PII, audit, retention)              │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ Layer 2: Platform Policy (Authorization & Quotas)           │
│ - Team permission levels (Guided, Advanced, Expert)         │
│ - Capacity quotas (max RPS, data size, retention)          │
│ - Resource limits (per namespace, per team)                │
│ - Compliance requirements (PCI, SOC2, GDPR)                │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ Layer 3: Pattern Selection (Platform Intelligence)          │
│ - Select patterns based on "needs" + policy                │
│ - Validate pattern compatibility                            │
│ - Calculate resource requirements                           │
│ - Choose backend slot assignments                           │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ Layer 4: Backend Binding (Operator-Managed)                 │
│ - Global backend registry (Kafka, Postgres, Redis, etc.)   │
│ - Backend capabilities (interfaces supported)               │
│ - Connection configurations (hosts, auth, pools)            │
│ - Slot → Backend mappings                                   │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ Layer 5: Frontend Exposure (Operator-Managed)               │
│ - Global frontend registry (REST, GraphQL, gRPC)           │
│ - Route mappings (HTTP → gRPC)                              │
│ - Protocol translation configs                              │
│ - API compatibility layers                                  │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ Layer 6: Runtime Execution (Pattern Runners & Proxies)      │
│ - Pattern runner process lifecycle                          │
│ - Backend connections and health                            │
│ - Request routing and load balancing                        │
│ - Observability (metrics, traces, logs)                     │
└─────────────────────────────────────────────────────────────┘

Terminology Standardization

Configuration Terms

Canonical Term	Definition	Previous Terms	Example
pattern	The high-level data access pattern a namespace uses	api, client_api, access_pattern	`pattern: queue`
needs	Functional requirements declared by application owner	requirements, guarantees, constraints	`needs.durability: strong`
slot	A capability requirement that a pattern has	backend_requirement, dependency	`registry` slot in multicast-registry
backend	A physical data store with connection config	backend_instance, data_store	`kafka-prod`, `postgres-primary`
capability	High-level operation a backend supports	interface, feature, operation	`publish`, `subscribe`, `scan`
interface	Specific API contract a backend implements	backend_interface, slot_interface	`keyvalue_basic`, `pubsub_persistent`
frontend	An API protocol exposure for a pattern	api_binding, protocol_adapter	`confluent-schema-registry-rest`

Authorization Terms

Canonical Term	Definition	Previous Terms	Example
team	A group of users who own resources	group, org	`payments-team`
namespace	An isolated data resource owned by a team	dataset, topic, schema	`order-processing`
permission level	Coarse-grained capability grant for teams	access_level, tier	`Guided`, `Advanced`, `Expert`
policy	Fine-grained authorization rule	rule, permission, constraint	"payments-team can admin order-*"
quota	Resource limit per team or namespace	limit, cap, threshold	`max_write_rps: 50000`
compliance	Regulatory requirement category	regulatory, audit_category	`PCI`, `SOC2`, `GDPR`

Layer 1: User Request (Application Owner)

Who: Application developers, service owners Controls: What data access pattern they need, functional requirements, who can access Cannot Control: How it's implemented, which backend is used, resource allocation

Canonical Request Schema

# Namespace request submitted by application owner
namespace: order-processing
team: payments-team

# Pattern declaration (WHAT)
pattern: queue  # Options: queue, pubsub, keyvalue, reader, transact, multicast-registry

# Functional requirements (NEEDS)
needs:
  # Durability
  durability: strong              # strong | eventual | best-effort
  replay: enabled                 # enabled | disabled

  # Capacity (estimates)
  write_rps: 5000                 # Writes per second
  read_rps: 10000                 # Reads per second
  data_size: 100GB                # Total data size
  retention: 30days               # Retention period

  # Message characteristics
  max_message_size: 1MB           # Largest message
  ordered: true                   # Preserve order

  # Consistency
  consistency: strong             # strong | eventual | bounded_staleness

# Access control (WHO)
access:
  owners:
    - team: payments-team
  consumers:
    - service: order-api         # Read-write
      permissions: [read, write]
    - service: analytics-pipeline # Read-only
      permissions: [read]

# Compliance & policies (POLICIES)
policies:
  pii_handling: enabled           # PII detection/encryption
  audit_logging: enabled          # Log all access
  compliance: [pci, sox]          # Compliance categories

What Users Control

✅ Pattern type: queue, pubsub, keyvalue, reader, transact ✅ Functional needs: Durability, consistency, capacity estimates ✅ Access control: Team ownership, consumer services ✅ Compliance: PII handling, audit requirements

❌ Pattern selection: Platform decides which patterns to apply ❌ Backend selection: Platform decides Kafka vs NATS vs Postgres ❌ Resource allocation: Platform calculates partitions, connections, replicas ❌ Network config: Platform manages VPCs, DNS, TLS

Layer 2: Platform Policy (Authorization & Quotas)

Who: Platform operators, SREs Controls: Team permissions, resource quotas, compliance enforcement Cannot Control: Individual namespace configs (managed by teams)

Permission Levels (ADR-002 Clarified)

Level 1: Guided (Default for All Teams)

permission_level: guided

# What teams can declare in namespace requests
allowed_config:
  patterns: [queue, pubsub, keyvalue, reader, transact]  # Standard patterns
  needs:
    durability: [strong, eventual, best-effort]
    consistency: [strong, eventual]
    write_rps: 0..50000              # Up to 50k RPS
    retention: 0..90days             # Up to 90 days
    max_message_size: 0..10MB        # Up to 10MB

# What's restricted
restricted_config:
  - backend_preference               # Cannot hint backend choice
  - backend_tuning                   # No low-level tuning
  - cross_region_replication         # No multi-region
  - custom_patterns                  # No pattern customization

Level 2: Advanced (Requires Platform Approval)

permission_level: advanced

# Additional capabilities beyond Guided
additional_config:
  needs:
    write_rps: 0..500000             # Up to 500k RPS
    retention: 0..365days            # Up to 1 year
    max_message_size: 0..100MB       # Up to 100MB
  backend_preference: [kafka, nats, postgres]  # Can hint (not guarantee)
  advanced_patterns: [multicast-registry, event-sourcing]

Level 3: Expert (Platform Team Only)

permission_level: expert

# Full control
additional_config:
  write_rps: unlimited
  retention: unlimited
  backend_override: true             # Force specific backend
  pattern_override: true             # Manually select patterns
  backend_tuning: true               # Direct backend params
  cross_region_replication: true
  custom_patterns: true

Team Quotas

# Team-level resource quotas (enforced at Layer 2)
team: payments-team
permission_level: advanced

quotas:
  # Aggregate limits across all namespaces
  max_namespaces: 50
  max_total_write_rps: 500000        # Sum across all namespaces
  max_total_data_size: 10TB
  max_retention_days: 365

  # Cost limits
  max_monthly_cost: 50000            # USD

  # Per-namespace limits
  max_namespace_write_rps: 100000
  max_namespace_data_size: 1TB

Topaz Policy Integration (ADR-050 Clarified)

Platform policies are implemented via Topaz for fine-grained enforcement:

# Policy: Enforce permission levels
package prism.config.authorization

# Default deny
default allow_config = false

# Guided teams: Can only use standard patterns
allow_config {
    input.team.permission_level == "guided"
    input.config.pattern in ["queue", "pubsub", "keyvalue", "reader", "transact"]
    input.config.needs.write_rps <= 50000
    input.config.needs.retention_days <= 90
}

# Advanced teams: Can use advanced patterns and higher limits
allow_config {
    input.team.permission_level == "advanced"
    input.config.needs.write_rps <= 500000
    input.config.needs.retention_days <= 365
}

# Expert teams: No restrictions
allow_config {
    input.team.permission_level == "expert"
}

# Quota enforcement
deny_config["quota_exceeded"] {
    team_total_rps := sum([ns.write_rps | ns := data.namespaces[input.team.name][_]])
    team_total_rps + input.config.needs.write_rps > input.team.quotas.max_total_write_rps
}

Layer 3: Pattern Selection (Platform Intelligence)

Who: Prism platform (automated) Controls: Which patterns to apply, how to compose them Based On: User needs + team policies

Pattern Selection Algorithm

# Platform automatically selects patterns based on declared needs
def select_patterns(pattern_type, needs, permission_level):
    patterns = []

    # Rule 1: Durability requirement
    if needs.durability == "strong":
        patterns.append({
            "type": "wal",
            "config": {"fsync": True, "batch_size": 100}
        })

    # Rule 2: Large message handling
    if needs.max_message_size > "1MB":
        patterns.append({
            "type": "claim-check",
            "config": {"threshold": "1MB", "storage": "s3"}
        })

    # Rule 3: Transactional consistency
    if needs.consistency == "strong" and pattern_type in ["queue", "transact"]:
        patterns.append({
            "type": "outbox",
            "config": {"database": "postgres"}
        })

    # Rule 4: Replay capability
    if needs.replay == "enabled":
        patterns.append({
            "type": "replay-store",
            "config": {"retention_days": needs.retention}
        })

    # Rule 5: Long retention with cost optimization
    if needs.retention > "30days":
        patterns.append({
            "type": "tiered-storage",
            "config": {
                "hot_tier_days": 7,
                "warm_tier_days": 30,
                "cold_tier": "s3"
            }
        })

    return patterns

# Example: User requests durable queue with large messages
needs = {
    "durability": "strong",
    "max_message_size": "50MB",
    "write_rps": 5000
}

patterns = select_patterns("queue", needs, "guided")
# Result: [WAL, ClaimCheck]
# Platform selected these automatically - user never specified them

Backend Selection Algorithm

# Platform automatically selects backend based on pattern + needs
def select_backend(pattern_type, patterns, needs, available_backends):
    if pattern_type == "keyvalue":
        if needs.read_rps > 100000:
            return "redis"  # High read throughput
        elif needs.data_size > "100GB":
            return "postgres"  # Large datasets
        else:
            return "redis"  # Default

    elif pattern_type == "pubsub":
        if "claim-check" in [p["type"] for p in patterns]:
            return "kafka"  # Handles large message references well
        elif needs.write_rps > 50000:
            return "kafka"  # High throughput
        else:
            return "nats"  # Lightweight, low latency

    elif pattern_type == "queue":
        if needs.ordered:
            return "kafka"  # Strong ordering guarantees
        elif "outbox" in [p["type"] for p in patterns]:
            return "postgres"  # Transactional outbox needs database
        else:
            return "kafka"  # Default

    elif pattern_type == "transact":
        return "postgres"  # Only backend with ACID transactions

    elif pattern_type == "multicast-registry":
        return {
            "registry": "postgres",    # Slot: registry
            "messaging": "kafka"        # Slot: messaging
        }

Layer 4: Backend Binding (Operator-Managed)

Who: Platform operators Controls: Global backend registry, connection configs, slot mappings Referenced By: Pattern runners at startup

Backend Registry (RFC-039 Clarified)

# Global backend registry (operator-managed)
backends:
  - name: kafka-prod
    type: kafka
    config:
      brokers: [kafka-01:9092, kafka-02:9092, kafka-03:9092]
      security_protocol: SASL_SSL
      sasl_mechanism: SCRAM-SHA-512
    capabilities: [publish, subscribe, consume_group, offset_commit]
    interfaces: [pubsub_basic, pubsub_persistent, pubsub_wildcards]
    metadata:
      region: us-west-2
      environment: production
      owner: platform-team

  - name: postgres-primary
    type: postgres
    config:
      host: postgres.prod.example.com
      port: 5432
      database: prism
      max_connections: 100
      ssl_mode: require
    capabilities: [get, set, scan, delete, transact]
    interfaces: [keyvalue_basic, keyvalue_scan, keyvalue_transactional, sql_query]
    metadata:
      region: us-west-2
      environment: production
      owner: platform-team

  - name: redis-cache
    type: redis
    config:
      host: redis.prod.example.com
      port: 6379
      database: 0
      pool_size: 50
    capabilities: [get, set, delete, ttl, scan]
    interfaces: [keyvalue_basic, keyvalue_ttl, keyvalue_scan, pubsub_basic]
    metadata:
      region: us-west-2
      environment: production
      owner: platform-team

Slot Binding (MEMO-006 Integration)

When platform selects backends for a pattern, it validates interface compatibility:

# Pattern: multicast-registry has 2 slots
pattern: multicast-registry
version: v1

slots:
  registry:
    description: "Stores identity → metadata mappings"
    required_interfaces: [keyvalue_basic, keyvalue_scan]
    optional_interfaces: [keyvalue_ttl]

  messaging:
    description: "Delivers multicast messages"
    required_interfaces: [pubsub_basic]
    optional_interfaces: [pubsub_persistent]

# Platform assigns backends to slots
slot_bindings:
  registry: postgres-primary    # Has keyvalue_basic + keyvalue_scan ✓
  messaging: kafka-prod          # Has pubsub_basic + pubsub_persistent ✓

Layer 5: Frontend Exposure (Operator-Managed)

Who: Platform operators Controls: API protocol mappings, REST routes, compatibility layers Enables: Non-gRPC clients (REST, GraphQL, Confluent Schema Registry)

Frontend Registry (RFC-039 Clarified)

# Global frontend registry (operator-managed)
frontends:
  - name: confluent-schema-registry-rest
    type: rest
    config:
      base_path: /
      enable_cors: true
      allowed_origins: ["*"]
    routes:
      # POST /subjects/{subject}/versions → RegisterSchema gRPC
      - http_method: POST
        path_pattern: /subjects/{subject}/versions
        pattern_rpc: prism.patterns.RegistryService.RegisterSchema
        param_mappings:
          - source: path
            source_name: subject
            target_field: subject
            required: true
          - source: body
            source_name: schema
            target_field: schema
            required: true
    metadata:
      api_version: v7
      compatibility: confluent

  - name: default-grpc
    type: grpc
    config:
      port: 8980
    metadata:
      description: Default gRPC interface for all patterns

Namespace Frontend Binding

# Namespace can enable multiple frontends
namespace: schema-registry
pattern: registry

slot_bindings:
  registry: postgres-primary

# Enable both REST (Confluent API) and gRPC (native)
frontend_bindings:
  - name: confluent-schema-registry-rest
    enabled: true
  - name: default-grpc
    enabled: true

Layer 6: Runtime Execution

Who: Pattern runners, proxies (automated) Controls: Process lifecycle, connections, request routing Observes: Metrics, traces, logs

Pattern Runner Startup Flow

1. Launcher receives pattern assignment from admin
2. Pattern runner starts, reads slot_bindings
3. For each slot:
   a. Fetch backend config from admin registry
   b. Validate backend implements required interfaces
   c. Create slot implementation (Kafka driver, Postgres driver, etc.)
   d. Connect to backend
4. Register with proxy for request routing
5. Begin processing requests

Configuration Hierarchy and Precedence

Precedence Rules

When multiple configuration sources could conflict, this is the precedence order (highest to lowest):

Platform Policy (Layer 2) - Hard limits, quotas, compliance requirements
- Example: Team quota is 50k RPS → Namespace cannot request 100k RPS
User Request (Layer 1) - Explicit namespace configuration
- Example: User sets needs.durability: strong → Platform must honor this
Pattern Defaults (Layer 3) - Pattern-specific default behavior
- Example: Queue pattern defaults to ordered delivery
Backend Capabilities (Layer 4) - Physical backend limitations
- Example: Kafka topic has max message size → Triggers claim-check pattern
Global Defaults (Layer 6) - System-wide default settings
- Example: Default TTL, default timeouts

Override Rules

❌ User CANNOT Override:

Team quotas (enforced by Layer 2)
Permission levels (set by platform)
Backend selection (platform-controlled)
Pattern composition (platform-controlled)

⚠️ User CAN Request (Subject to Policy):

Backend preference (if permission level allows)
Capacity estimates (up to team quota)
Retention period (within policy limits)

✅ User CONTROLS:

Pattern type selection
Functional needs declaration
Access control policies
Compliance requirements

Multi-Tenancy Model

Hierarchy: Tenant → Team → Namespace

┌──────────────────────────────────────────┐
│ Tenant (Organization)                    │
│ - Acme Corp                              │
│                                           │
│  ┌────────────────────────────────────┐  │
│  │ Team (Engineering Unit)            │  │
│  │ - payments-team                    │  │
│  │ - user-platform-team               │  │
│  │                                     │  │
│  │  ┌──────────────────────────────┐  │  │
│  │  │ Namespace (Data Resource)    │  │  │
│  │  │ - order-processing           │  │  │
│  │  │ - payment-transactions       │  │  │
│  │  │ - user-profiles              │  │  │
│  │  └──────────────────────────────┘  │  │
│  └────────────────────────────────────┘  │
└──────────────────────────────────────────┘

Relationships

Tenant: Root organization (e.g., "Acme Corp")
- Has global quotas and compliance requirements
- Multiple teams belong to one tenant
Team: Engineering unit (e.g., "payments-team")
- Has permission level (Guided, Advanced, Expert)
- Has team-level quotas (sum across namespaces)
- Owns multiple namespaces
Namespace: Isolated data resource (e.g., "order-processing")
- Owned by exactly one team
- Has pattern + needs configuration
- Consumes part of team quota

Cross-Namespace Patterns

Shared Backends (Allowed):

# Both namespaces use same backend
namespace: order-processing
slot_bindings:
  messaging: kafka-prod

namespace: user-notifications
slot_bindings:
  messaging: kafka-prod  # Same backend, different topic

Cross-Namespace Communication (Via PubSub):

# Producer namespace
namespace: order-processing
pattern: queue
access:
  producers: [order-api]

# Consumer namespace
namespace: analytics-ingest
pattern: pubsub
access:
  subscribers: [analytics-pipeline]
# Platform routes: order-processing → analytics-ingest

Cross-Team Access (Restricted):

# Default: Teams cannot access each other's namespaces
# Exception: Explicit cross-team grant via Topaz policy

# Policy: Allow analytics-team to read from payments-team namespace
namespace: order-processing
team: payments-team
access:
  consumers:
    - team: analytics-team
      permissions: [read]
      approved_by: platform-team  # Requires platform approval

Role-Based Configuration Responsibilities

Application Owner (User)

Can Do:

Request new namespace
Declare pattern and needs
Set access control policies
Specify compliance requirements
Update capacity estimates
Add/remove consumer services

Cannot Do:

Change team permission level
Override team quotas
Select backend implementations
Configure low-level backend settings
Modify platform policies

Example Workflow:

Create namespace request YAML
Submit via prism namespace create --file request.yaml
Platform validates against team policy
Platform provisions namespace
Use namespace via SDK

Platform Operator (Operator)

Can Do:

Manage backend registry (add/update/delete backends)
Manage frontend registry (add/update/delete frontends)
Set team permission levels
Configure team quotas
Define compliance policies
Update platform-wide defaults

Cannot Do:

Override user's declared needs (within policy)
Modify namespace access control (team-owned)
Change namespace pattern (team-owned)

Example Workflow:

Register new backend: prism-admin backend register kafka-prod ...
Set team permission level: prism-admin team update payments-team --level advanced
Platform automatically uses new backend for new namespaces
Existing namespaces continue using old backend until migration

Platform Developer (Developer)

Can Do:

Implement new patterns
Add new backend drivers
Define slot interfaces (MEMO-006)
Create frontend adapters
Update pattern selection algorithms

Cannot Do:

Change user-facing configuration schemas (breaking changes)
Modify production backend configs directly
Override runtime policies

Example Workflow:

Implement new pattern with slot requirements
Define slot interfaces in pattern schema
Platform automatically validates backends against slots
Pattern becomes available for users to request

Configuration Evolution and Versioning

Namespace Lifecycle

Creation:

# v1: Initial creation
namespace: order-processing
pattern: queue
needs:
  write_rps: 5000

Evolution (Non-Breaking):

# v2: Increase capacity (within quota)
namespace: order-processing
pattern: queue  # Unchanged
needs:
  write_rps: 10000  # Increased
# Platform: Add partitions, no client disruption

Evolution (Breaking):

# v3: Change pattern (requires migration)
namespace: order-processing
pattern: multicast-registry  # Changed from queue
# Platform: Create new pattern runner, migrate data, update routing
# User: Update client SDK to use new API

Backend Evolution

Backward Compatible:

# Backend config change (transparent to users)
backend: kafka-prod
config:
  brokers: [kafka-04:9092, kafka-05:9092, kafka-06:9092]  # New brokers
# Pattern runners reconnect on restart
# Existing connections continue working

Breaking Change:

# Backend replacement (requires migration)
backend: kafka-prod-v2  # New backend
type: kafka
# Operator must:
# 1. Register new backend
# 2. Update slot_bindings in namespaces
# 3. Restart pattern runners
# 4. Migrate data (if needed)

Frontend Evolution

Backward Compatible:

# Add new route (existing routes continue working)
frontend: confluent-schema-registry-rest
routes:
  - path_pattern: /subjects/{subject}/versions  # Existing
  - path_pattern: /subjects  # New route added

Breaking Change:

# Change route mapping (clients must update)
frontend: confluent-schema-registry-rest-v2  # New frontend
# Operator must:
# 1. Register new frontend version
# 2. Update namespace frontend_bindings
# 3. Clients migrate to new routes

Examples: End-to-End Configuration Flows

Example 1: Simple Key-Value Cache

User Request:

namespace: user-profiles-cache
team: user-platform-team
pattern: keyvalue
needs:
  durability: eventual
  read_rps: 50000
  write_rps: 500
  ttl: 15min
access:
  owners: [user-platform-team]
  consumers: [user-api, user-profile-service]

Platform Policy Check:

team: user-platform-team
permission_level: guided
quotas:
  max_total_write_rps: 100000  # Currently using 30k
  # 500 RPS request is within quota ✓

Platform Pattern Selection:

# Algorithm sees:
# - pattern: keyvalue
# - needs.read_rps: 50000 (high)
# - needs.durability: eventual
# - needs.ttl: 15min

patterns = []  # No additional patterns needed for simple cache
backend = "redis-cache"  # High read throughput + TTL support

Backend Binding:

# Platform assigns backend
namespace: user-profiles-cache
pattern: keyvalue
slot_bindings:
  keyvalue: redis-cache
# Redis backend implements: keyvalue_basic, keyvalue_ttl ✓

Result:

User gets fast Redis-backed cache
Platform selected Redis automatically (user didn't specify)
TTL handled natively by Redis
No additional patterns needed

Example 2: Durable Queue with Large Messages

User Request:

namespace: video-uploads
team: media-team
pattern: queue
needs:
  durability: strong
  max_message_size: 50MB
  write_rps: 1000
  retention: 90days
access:
  owners: [media-team]
  consumers: [video-processor, thumbnail-generator]

Platform Policy Check:

team: media-team
permission_level: guided
quotas:
  max_namespace_write_rps: 50000  # 1000 RPS is within limit ✓
  max_namespace_data_size: 1TB    # 90 days retention is within limit ✓

Platform Pattern Selection:

# Algorithm sees:
# - needs.durability: strong → Add WAL pattern
# - needs.max_message_size: 50MB → Add ClaimCheck pattern
# - needs.retention: 90days → Add TieredStorage pattern

patterns = [
    {"type": "wal", "config": {"fsync": True}},
    {"type": "claim-check", "config": {"threshold": "1MB", "storage": "s3"}},
    {"type": "tiered-storage", "config": {"hot_tier_days": 7}}
]
backend = "kafka-prod"

Backend Binding:

namespace: video-uploads
pattern: queue
slot_bindings:
  queue: kafka-prod
  durability: postgres-primary  # For WAL
  object_storage: s3-prod        # For ClaimCheck

Result:

User gets durable queue with large message support
Platform added 3 patterns automatically
Platform assigned 3 backends (Kafka, Postgres, S3)
User code just calls publish() - patterns are transparent

Example 3: Schema Registry with REST API

User Request:

namespace: schema-registry
team: platform-team
pattern: registry
needs:
  durability: strong
  consistency: strong
  write_rps: 100
  read_rps: 1000
access:
  owners: [platform-team]
  consumers: [all-services]  # Global access
policies:
  audit_logging: enabled

Platform Policy Check:

team: platform-team
permission_level: expert  # Platform team has full control
# No quota checks for expert level

Platform Pattern Selection:

# Algorithm sees:
# - pattern: registry
# - needs.consistency: strong

patterns = []  # Registry pattern is self-contained
backend = "postgres-primary"  # Strong consistency requires SQL

Backend Binding:

namespace: schema-registry
pattern: registry
slot_bindings:
  registry: postgres-primary
  cache: redis-cache  # Optional caching

Frontend Binding (Operator adds Confluent compatibility):

namespace: schema-registry
frontend_bindings:
  - name: confluent-schema-registry-rest
    enabled: true
  - name: default-grpc
    enabled: true

Result:

Registry pattern stores schemas in PostgreSQL
Confluent REST API exposed via frontend binding
Native gRPC also available
Existing Confluent clients work without modification

Migration from Current State

This RFC consolidates existing designs - no breaking changes required:

Terminology Updates

Find and Replace across docs:

"api:" → "pattern:"
"requirements:" → "needs:"
"client_api:" → "pattern:"
"access_pattern:" → "pattern:"

Policy Consolidation

Merge permission models:

ADR-002 permission levels (Guided, Advanced, Expert) → Keep as primary model
ADR-050 Topaz policies → Implement enforcement for permission levels
RFC-027 team quotas → Integrate into permission level limits

Configuration Schema Updates

Add canonical fields to namespace config:

message NamespaceConfig {
  string namespace = 1;
  string team = 2;

  // OLD (deprecated but supported)
  string client_api = 3 [deprecated = true];
  map<string, string> requirements = 4 [deprecated = true];

  // NEW (canonical)
  string pattern = 5;              // Replaces client_api
  NamespaceNeeds needs = 6;        // Replaces requirements map

  // Unchanged
  AccessControl access = 7;
  Policies policies = 8;
  map<string, string> slot_bindings = 9;
  repeated FrontendBinding frontend_bindings = 10;
}

Consolidated by This RFC

ADR-002: Client-Originated Configuration
ADR-006: Namespace and Multi-Tenancy
ADR-007: Authentication and Authorization
ADR-022: Dynamic Client Configuration
ADR-050: Topaz Policy Authorization
RFC-014: Layered Data Access Patterns
RFC-027: Namespace Configuration Client Perspective
RFC-039: Backend Configuration Registry
MEMO-006: Backend Interface Decomposition

Unchanged (Compatible)

ADR-001: Rust for Proxy
ADR-003: Protobuf as Single Source of Truth
ADR-005: Backend Plugin Architecture
RFC-008: Proxy Plugin Architecture
RFC-035: Pattern Process Launcher

Open Questions

Quota Enforcement Timing: When do we check team quotas - at request validation or at runtime allocation?
- Proposed: Validation time (fail fast if quota would be exceeded)
Cross-Tenant Resource Sharing: Can teams from different tenants share backends?
- Proposed: Phase 2 feature - initially, backends are tenant-scoped
Configuration Audit Trail: How do we track who changed what and when?
- Proposed: Admin Raft log already provides audit trail, expose via API
Dynamic Policy Updates: Can platform operators update policies without restarting?
- Proposed: Yes - Topaz supports live policy updates via Git sync
Namespace Migration: How do users migrate namespaces between patterns (queue → multicast-registry)?
- Proposed: Manual migration workflow with platform team assistance

Success Criteria

✅ Terminology Consistency: All docs use "pattern", "needs", "slot", "backend", "frontend"
✅ Clear Hierarchy: Six-layer model documented with precedence rules
✅ Role Clarity: User/Operator/Developer responsibilities explicitly defined
✅ Authorization Model: Permission levels + Topaz policies + quotas integrated
✅ No Breaking Changes: Existing namespace configs continue working
✅ Migration Path: Deprecated fields supported, new canonical fields added

Revision History

2025-11-14: Initial RFC consolidating configuration model across all documents

Abstract​

Motivation​

Current State: Configuration Across Multiple Documents​

Problems​

Goals​

Non-Goals​

Unified Configuration Model​

Core Principles​

Configuration Layers (Top to Bottom)​

Terminology Standardization​

Configuration Terms​

Authorization Terms​

Layer 1: User Request (Application Owner)​

Canonical Request Schema​

What Users Control​

Layer 2: Platform Policy (Authorization & Quotas)​

Permission Levels (ADR-002 Clarified)​

Team Quotas​

Topaz Policy Integration (ADR-050 Clarified)​

Layer 3: Pattern Selection (Platform Intelligence)​

Pattern Selection Algorithm​

Backend Selection Algorithm​

Layer 4: Backend Binding (Operator-Managed)​

Backend Registry (RFC-039 Clarified)​

Slot Binding (MEMO-006 Integration)​

Layer 5: Frontend Exposure (Operator-Managed)​

Frontend Registry (RFC-039 Clarified)​

Namespace Frontend Binding​

Layer 6: Runtime Execution​

Pattern Runner Startup Flow​

Configuration Hierarchy and Precedence​

Precedence Rules​

Override Rules​

Multi-Tenancy Model​

Hierarchy: Tenant → Team → Namespace​

Relationships​

Cross-Namespace Patterns​

Role-Based Configuration Responsibilities​

Application Owner (User)​

Platform Operator (Operator)​

Platform Developer (Developer)​

Configuration Evolution and Versioning​

Namespace Lifecycle​

Backend Evolution​

Frontend Evolution​

Examples: End-to-End Configuration Flows​

Example 1: Simple Key-Value Cache​

Example 2: Durable Queue with Large Messages​

Example 3: Schema Registry with REST API​

Migration from Current State​

Terminology Updates​

Policy Consolidation​

Configuration Schema Updates​

Related Documents​

Consolidated by This RFC​

Unchanged (Compatible)​

Open Questions​

Success Criteria​

Revision History​

Abstract

Motivation

Current State: Configuration Across Multiple Documents

Problems

Goals

Non-Goals

Unified Configuration Model

Core Principles

Configuration Layers (Top to Bottom)

Terminology Standardization

Configuration Terms

Authorization Terms

Layer 1: User Request (Application Owner)

Canonical Request Schema

What Users Control

Layer 2: Platform Policy (Authorization & Quotas)

Permission Levels (ADR-002 Clarified)

Team Quotas

Topaz Policy Integration (ADR-050 Clarified)

Layer 3: Pattern Selection (Platform Intelligence)

Pattern Selection Algorithm

Backend Selection Algorithm

Layer 4: Backend Binding (Operator-Managed)

Backend Registry (RFC-039 Clarified)

Slot Binding (MEMO-006 Integration)

Layer 5: Frontend Exposure (Operator-Managed)

Frontend Registry (RFC-039 Clarified)

Namespace Frontend Binding

Layer 6: Runtime Execution

Pattern Runner Startup Flow

Configuration Hierarchy and Precedence

Precedence Rules

Override Rules

Multi-Tenancy Model

Hierarchy: Tenant → Team → Namespace

Relationships

Cross-Namespace Patterns

Role-Based Configuration Responsibilities

Application Owner (User)

Platform Operator (Operator)

Platform Developer (Developer)

Configuration Evolution and Versioning

Namespace Lifecycle

Backend Evolution

Frontend Evolution

Examples: End-to-End Configuration Flows

Example 1: Simple Key-Value Cache

Example 2: Durable Queue with Large Messages

Example 3: Schema Registry with REST API

Migration from Current State

Terminology Updates

Policy Consolidation

Configuration Schema Updates

Related Documents

Consolidated by This RFC

Unchanged (Compatible)

Open Questions

Success Criteria

Revision History