Practice — design a multi-region active-active architecture

This practice exercise applies the networking concepts from Module 1 to a realistic design challenge. You will architect a globally available scorecard platform that serves IPL match data to viewers across India, Southeast Asia, and the UK with sub-200ms latency, zero-downtime deployments, and automatic regional failover. The scenario integrates every major service covered so far: Route 53, CloudFront, ALB, VPC networking, and the AWS Well-Architected Framework.

An active-active architecture runs the full application stack in two or more AWS regions simultaneously, with all regions serving live traffic at all times. This is distinct from active-passive, where the secondary region only receives traffic after the primary fails. Active-active requires all regions to share state, route traffic intelligently based on proximity, and handle partial failures gracefully without a global failover event that disrupts all users simultaneously.

Work through each design task in sequence. For each task, sketch the architecture first, then validate it against the Well-Architected Framework pillar most relevant to that task: Reliability for failover design, Performance Efficiency for latency routing, Cost Optimization for CDN caching, and Security for cross-region access controls. The exercise ends with two quiz questions that test your ability to apply the design principles to novel failure scenarios.

Scenario

The IPL Scorecard Platform serves live match data, player statistics, and video highlights to 8 million concurrent viewers during finals matches. The platform currently runs in ap-south-1 (Mumbai) only. After a scheduled maintenance window caused a two-hour outage during a semi-final, the product team requires a multi-region architecture that can survive a complete regional failure without any manual operator intervention and with a maximum data loss window of 60 seconds.

The application has three distinct traffic profiles that require different architectural treatment. Live score API endpoints at /api/scores/* are write-heavy during matches, with updates every 3 seconds from stadium data feeds, and must be strongly consistent across regions. Player statistics at /api/players/* are read-heavy and can tolerate 60-second eventual consistency. Static assets at /static/* including team logos and UI components never change during a match and can be cached aggressively for 24 hours.

The engineering team has eight weeks to deliver the first milestone: multi-region read availability with automatic DNS failover. Full active-active write consistency using DynamoDB Global Tables is scheduled for the second milestone. This practice exercise focuses on the networking and DNS layer of the first milestone: designing the traffic routing, health check, and CDN caching architecture that makes both milestones operationally sound.

Architecture Requirements

• Deploy the application stack in ap-south-1 (primary) and ap-southeast-1 (secondary) with identical infrastructure in both regions.
• Route 53 latency routing directs each viewer to the closest healthy region; health checks use calculated checks aggregating three AZ-level checks per region with a threshold of 2 of 3.
• CloudFront distribution with three cache behaviours: /api/scores/* with CachingDisabled and Lambda@Edge JWT validation, /api/players/* with 60-second TTL, and /static/* with 86400-second TTL.
• ALB in each region uses path-based routing to separate the scores microservice and the players microservice into independent target groups, enabling independent scaling and independent health check evaluation.
• VPC design uses Transit Gateway in each region with a dedicated inspection VPC hosting Network Firewall; all cross-service traffic routes through the inspection VPC.
• Zero-downtime deployment uses ALB weighted target groups for blue/green traffic shifts: 10% canary for 10 minutes, then 50% for 10 minutes, then 100% cutover if error rate stays below 0.1%.
• DNS TTL of 60 seconds on all failover-eligible records; DNS TTL of 3600 seconds on CloudFront distribution CNAME and static asset records.

Design Task 1 — Route 53 and Health Check Architecture

Design the Route 53 record set for api.ipl.srihayavadhana.in that routes viewers to the closest healthy region. Your design must handle three failure scenarios: a single AZ failure in Mumbai (should not trigger failover), all three AZs failing in Mumbai (should trigger failover within 90 seconds), and CloudFront origin group failover when the primary ALB returns 5xx (should trigger within 60 seconds of the ALB’s health check detecting failure).

Consider how EvaluateTargetHealth interacts with your calculated health check design. The ALB’s built-in target health evaluation runs every 30 seconds with a threshold of 2 unhealthy consecutive checks, meaning the ALB marks a target unhealthy after approximately 60 seconds. Your Route 53 health check runs every 30 seconds with a failure threshold of 3. Draw the timeline from a target failure event to the first DNS response that excludes the Mumbai record.

# Starter: Route 53 latency routing with calculated health checks.
# Complete the TODO sections as part of the exercise.
import boto3
r53 = boto3.client('route53')

# TODO 1: Create AZ-level health checks for Mumbai (ap-south-1).
# Each should check the ALB health check endpoint on port 443.
# Endpoint: https://ipl-alb-az{n}.ap-south-1.elb.amazonaws.com/health

# TODO 2: Create a calculated health check aggregating the three AZ checks.
# Threshold: 2 of 3 must pass for the region to be considered healthy.

# Reference: latency routing record with calculated health check.
r53.change_resource_record_sets(
    HostedZoneId='/hostedzone/IPLZONEID',
    ChangeBatch={'Changes': [{
        'Action': 'UPSERT',
        'ResourceRecordSet': {
            'Name': 'api.ipl.srihayavadhana.in', 'Type': 'A',
            'SetIdentifier': 'mumbai-latency',
            'Region': 'ap-south-1',
            'HealthCheckId': 'TODO_CALCULATED_HC_ID',
            'AliasTarget': {
                'HostedZoneId': 'ZP97RAFLXTNZK',
                'DNSName': 'ipl-scorecard-alb.ap-south-1.elb.amazonaws.com',
                'EvaluateTargetHealth': True,
            },
        },
    }]}
)
print('Latency record with health check configured.')

Design Task 2 — CloudFront Cache Behaviour Architecture

Design the CloudFront distribution with three cache behaviours for the three traffic profiles. For /api/scores/*, justify why CachingDisabled is correct even though it means every viewer request hits the origin: the answer lies in the 3-second update interval and the strong consistency requirement. For /api/players/*, calculate the cache hit rate improvement from a 60-second TTL given that player stats are queried by 8 million viewers for approximately 200 players, and each stat is updated at most once per over (every 6 deliveries, roughly every 2 minutes).

Design the Lambda@Edge function placement for JWT validation. The validation function must execute before the cache is checked to prevent unauthenticated users from receiving cached responses intended for authenticated users. Map the function to the correct CloudFront trigger point and explain why attaching it to the origin request trigger instead would create a security vulnerability where cached responses from one authenticated user could be served to an unauthenticated user on a subsequent cache hit for the same URL.

# Starter: CloudFront distribution skeleton.
# Complete the CacheBehaviors and FunctionAssociations sections.
import boto3
cf = boto3.client('cloudfront')

dist_config = {
    'Comment': 'IPL Scorecard Platform — Multi-Region',
    'DefaultCacheBehavior': {
        'TargetOriginId': 'ipl-api-origin-group',  # origin group with Mumbai + Singapore
        'ViewerProtocolPolicy': 'redirect-to-https',
        'CachePolicyId': '4135ea2d-6df8-44a3-9df3-4b5a84be39ad',  # CachingDisabled
        # TODO: Add Lambda@Edge JWT validation at the correct trigger.
    },
    'CacheBehaviors': {
        'Quantity': 2,
        'Items': [
            # TODO: /api/players/* with 60-second TTL.
            # TODO: /static/* with 86400-second TTL and Compress=True.
        ],
    },
    'Origins': {'Quantity': 1, 'Items': [
        # TODO: Configure origin group with Mumbai ALB as primary and Singapore ALB as failover.
    ]},
    'Enabled': True,
    'PriceClass': 'PriceClass_All',
}
print('Distribution config skeleton ready — complete the TODOs.')

Design Task 3 — Blue/Green Deployment Runbook

Design the ALB weighted target group runbook for zero-downtime deployment of the scores microservice. The runbook must specify the exact sequence of API calls to shift traffic, the CloudWatch metrics to monitor at each stage, the error rate threshold that triggers an automatic rollback, and the minimum observation window at each traffic percentage before advancing to the next stage. The runbook should be executable by an on-call engineer under time pressure without referencing external documentation.

# Starter: Blue/green deployment runbook as executable code.
# Fill in the monitoring and rollback logic.
import boto3, time
elb = boto3.client('elbv2', region_name='ap-south-1')
cw  = boto3.client('cloudwatch', region_name='ap-south-1')

BLUE_TG  = 'arn:aws:elasticloadbalancing:ap-south-1:111:targetgroup/ipl-scores-blue/abc'
GREEN_TG = 'arn:aws:elasticloadbalancing:ap-south-1:111:targetgroup/ipl-scores-green/def'
LISTENER = 'arn:aws:elasticloadbalancing:ap-south-1:111:listener/app/ipl-alb/xyz/uvw'

def shift_traffic(blue_weight: int, green_weight: int):
    elb.modify_listener(
        ListenerArn=LISTENER,
        DefaultActions=[{'Type':'forward','ForwardConfig':{'TargetGroups':[
            {'TargetGroupArn': BLUE_TG,  'Weight': blue_weight},
            {'TargetGroupArn': GREEN_TG, 'Weight': green_weight},
        ]}}]
    )
    print(f'Traffic: blue={blue_weight}% green={green_weight}%')

def get_error_rate(tg_arn: str, minutes: int = 5) -> float:
    # TODO: Query CloudWatch HTTPCode_Target_5XX_Count and RequestCount
    # for the target group over the last {minutes} minutes.
    # Return error rate as a float between 0 and 1.
    pass

# TODO: Implement the deployment stages:
# Stage 1: 90/10 for 10 minutes; rollback if error rate > 0.001
# Stage 2: 50/50 for 10 minutes; rollback if error rate > 0.001
# Stage 3: 0/100 full cutover
print('Runbook skeleton ready — implement the staged rollout logic.')

Evaluation Criteria

• Route 53 design includes latency routing records in both regions, calculated health checks with 2-of-3 AZ threshold, and EvaluateTargetHealth=True on all Alias records pointing to ALBs.
• CloudFront distribution has three distinct cache behaviours with correct TTLs per traffic profile and Lambda@Edge JWT validation attached at the viewer-request trigger, not the origin-request trigger.
• The blue/green runbook specifies exact CloudWatch metrics, numeric error rate thresholds, and minimum observation windows at each traffic percentage stage, making it executable without interpretation.
• The architecture satisfies the Reliability pillar by achieving an RTO under 90 seconds for a full regional failure with no manual operator intervention required.
• The Well-Architected review of the design identifies at least three potential HRIs across the Security, Reliability, and Cost Optimization pillars, with a proposed remediation for each.
• The design explicitly addresses the JWT validation vulnerability: Lambda@Edge is attached at viewer-request, not origin-request, with a written explanation of the cache poisoning risk if attached at the wrong trigger.