Safely Migrate Databases at Millions of Requests per Second (Video)
info
AWS re:Invent 2023 conference talk featuring Netflix engineers discussing production database migration strategies.
Video
Presented by: AWS Events (Netflix Engineering Team)
Overview
This talk from AWS re:Invent 2023 covers Netflix's approach to safely migrating databases that handle millions of requests per second in production environments. The presentation provides real-world insights into the patterns and practices Netflix uses to achieve zero-downtime migrations at massive scale.
Key Topics Covered
The video discusses production-proven strategies for:
- Zero-downtime migrations: Techniques for migrating databases while maintaining service availability
- Dual-write patterns: Writing to both old and new datastores simultaneously during migration
- Shadow traffic validation: Testing new systems with production load before cutover
- Risk mitigation: Strategies for reducing blast radius and enabling quick rollbacks
- Scale challenges: Handling millions of requests per second during migration
- Operational practices: Monitoring, validation, and phased cutover approaches
Relevance to Prism
This video directly informs several Prism design decisions:
- ADR-009: Shadow Traffic Migrations: Implementing Netflix's shadow traffic pattern
- Dual-Write Pattern: Our documentation of the dual-write approach
- Data Evolution: Schema evolution and migration strategies
- RFC-009: Distributed Reliability Patterns: Incorporating migration resilience patterns
Related Resources
- Netflix Dual-Write Migration Pattern: Detailed documentation of the pattern
- Netflix Data Gateway Overview: Summary of Netflix Data Gateway and migration approaches
- Shadow Traffic ADR: Prism's implementation of shadow traffic testing
Source: AWS re:Invent 2023 - NFX307