Test Your AWS DR Plan with AWS Backup and Fault Injection Service

What you'll build

A cross-region disaster recovery pipeline for a DynamoDB table using AWS Backup, plus an AWS Fault Injection Service (FIS) experiment that severs your app's access to DynamoDB in the primary region. You'll run the restore in the DR region while the fault is live and walk away with measured RTO and RPO numbers instead of estimates.

Prerequisites

• An AWS account you can safely break things in. The FIS experiment swaps network ACLs on the subnets you target, so use a sandbox account, not one shared with production.
• AWS CLI v2 (verified against 2.36) with admin-level credentials configured.
• python3 on your PATH (used only for timestamp math at the end).
• Two regions. This tutorial uses us-east-1 as primary and us-west-2 as DR.
• A default VPC in the primary region (most accounts have one). If yours doesn't, substitute your app subnets' ARNs in step 5.
• Cost: cents for backup storage, plus FIS billing per action-minute. A 30-minute single-action experiment costs a few dollars. Everything is cleaned up at the end.

Verified against the AWS Backup and FIS docs current as of September 2026.

1. Create the workload

Set variables, create a table in the primary region, and seed a record you'll later prove survived the "outage":

export PRIMARY=us-east-1
export DR=us-west-2
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

aws dynamodb create-table \
  --table-name orders \
  --attribute-definitions AttributeName=id,AttributeType=S \
  --key-schema AttributeName=id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --tags Key=dr-tier,Value=critical \
  --region $PRIMARY

aws dynamodb wait table-exists --table-name orders --region $PRIMARY

aws dynamodb put-item --table-name orders \
  --item '{"id":{"S":"order-1001"},"total":{"N":"49.99"}}' \
  --region $PRIMARY

The dr-tier=critical tag is what the backup plan will select on, so protecting a new resource later means tagging it, not editing the plan.

2. Enable advanced DynamoDB backups and create the vaults

Cross-region copy of DynamoDB backups only works when AWS Backup fully manages them ("advanced features"). Accounts that adopted AWS Backup after November 2021 have this on by default, but set it explicitly because the order matters: backups taken before you enable it can never be copied.

aws backup update-region-settings \
  --resource-type-opt-in-preference DynamoDB=true \
  --resource-type-management-preference DynamoDB=true \
  --region $PRIMARY

aws backup create-backup-vault --backup-vault-name primary-vault --region $PRIMARY
aws backup create-backup-vault --backup-vault-name dr-vault --region $DR

aws iam create-role --role-name dr-backup-role \
  --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"backup.amazonaws.com"},"Action":"sts:AssumeRole"}]}'

aws iam attach-role-policy --role-name dr-backup-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup
aws iam attach-role-policy --role-name dr-backup-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForRestores

3. Create the backup plan with cross-region copy

The plan runs hourly and copies every recovery point to the DR vault via CopyActions:

cat > backup-plan.json <<EOF
{
  "BackupPlanName": "dr-plan",
  "Rules": [{
    "RuleName": "hourly-with-dr-copy",
    "TargetBackupVaultName": "primary-vault",
    "ScheduleExpression": "cron(0 * * * ? *)",
    "StartWindowMinutes": 60,
    "CompletionWindowMinutes": 180,
    "Lifecycle": {"DeleteAfterDays": 7},
    "CopyActions": [{
      "DestinationBackupVaultArn": "arn:aws:backup:${DR}:${ACCOUNT_ID}:backup-vault:dr-vault",
      "Lifecycle": {"DeleteAfterDays": 7}
    }]
  }]
}
EOF

PLAN_ID=$(aws backup create-backup-plan \
  --backup-plan file://backup-plan.json \
  --query BackupPlanId --output text --region $PRIMARY)

aws backup create-backup-selection --backup-plan-id $PLAN_ID --region $PRIMARY \
  --backup-selection '{"SelectionName":"critical-tier","IamRoleArn":"arn:aws:iam::'$ACCOUNT_ID':role/dr-backup-role","ListOfTags":[{"ConditionType":"STRINGEQUALS","ConditionKey":"dr-tier","ConditionValue":"critical"}]}'

4. Take a backup and copy it to the DR region

The plan handles this on schedule. To finish the tutorial without waiting for the top of the hour, run one on-demand backup and copy it yourself:

BACKUP_JOB=$(aws backup start-backup-job \
  --backup-vault-name primary-vault \
  --resource-arn arn:aws:dynamodb:$PRIMARY:$ACCOUNT_ID:table/orders \
  --iam-role-arn arn:aws:iam::$ACCOUNT_ID:role/dr-backup-role \
  --query BackupJobId --output text --region $PRIMARY)

until [ "$(aws backup describe-backup-job --backup-job-id $BACKUP_JOB \
  --query State --output text --region $PRIMARY)" = "COMPLETED" ]; do sleep 15; done

RP_ARN=$(aws backup describe-backup-job --backup-job-id $BACKUP_JOB \
  --query RecoveryPointArn --output text --region $PRIMARY)

COPY_JOB=$(aws backup start-copy-job \
  --recovery-point-arn $RP_ARN \
  --source-backup-vault-name primary-vault \
  --destination-backup-vault-arn arn:aws:backup:$DR:$ACCOUNT_ID:backup-vault:dr-vault \
  --iam-role-arn arn:aws:iam::$ACCOUNT_ID:role/dr-backup-role \
  --query CopyJobId --output text --region $PRIMARY)

until [ "$(aws backup describe-copy-job --copy-job-id $COPY_JOB \
  --query CopyJob.State --output text --region $PRIMARY)" = "COMPLETED" ]; do sleep 30; done

Cross-region copies are slow relative to the table size, so don't panic if the loop runs for a while even on this near-empty table.

5. Create the FIS role and experiment template

The experiment uses the aws:network:disrupt-connectivity action with scope: dynamodb, which clones each target subnet's network ACL, adds deny rules for the regional DynamoDB endpoint, and swaps it in for the duration. From inside those subnets, DynamoDB in us-east-1 is down. FIS restores the original ACLs when the experiment ends or you stop it.

aws iam create-role --role-name dr-fis-role \
  --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"fis.amazonaws.com"},"Action":"sts:AssumeRole"}]}'

aws iam attach-role-policy --role-name dr-fis-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSFaultInjectionSimulatorNetworkAccess

SUBNET_ARNS=$(aws ec2 describe-subnets \
  --filters Name=default-for-az,Values=true \
  --query 'Subnets[].SubnetArn' --output json --region $PRIMARY)

cat > fis-template.json <<EOF
{
  "description": "Block the regional DynamoDB endpoint for a DR game day",
  "stopConditions": [{"source": "none"}],
  "targets": {
    "app-subnets": {
      "resourceType": "aws:ec2:subnet",
      "resourceArns": $SUBNET_ARNS,
      "selectionMode": "ALL"
    }
  },
  "actions": {
    "block-dynamodb": {
      "actionId": "aws:network:disrupt-connectivity",
      "parameters": {"scope": "dynamodb", "duration": "PT30M"},
      "targets": {"Subnets": "app-subnets"}
    }
  },
  "roleArn": "arn:aws:iam::${ACCOUNT_ID}:role/dr-fis-role"
}
EOF

TEMPLATE_ID=$(aws fis create-experiment-template \
  --cli-input-json file://fis-template.json \
  --query experimentTemplate.id --output text --region $PRIMARY)

A real game day would add a CloudWatch alarm as a stop condition; "source": "none" keeps this focused.

6. Start the outage and run the DR restore

Record the moment the outage starts. That timestamp anchors both measurements: RPO is how far behind your newest DR copy was at that instant, RTO is how long until the restored table is live.

T0=$(date -u +%Y-%m-%dT%H:%M:%SZ)
EXP_ID=$(aws fis start-experiment --experiment-template-id $TEMPLATE_ID \
  --query experiment.id --output text --region $PRIMARY)

Now execute your DR runbook against the DR region only. Find the newest recovery point in the DR vault and restore it to a new table:

DR_RP=$(aws backup list-recovery-points-by-backup-vault \
  --backup-vault-name dr-vault --region $DR \
  --query 'sort_by(RecoveryPoints,&CreationDate)[-1].RecoveryPointArn' --output text)

RESTORE_JOB=$(aws backup start-restore-job \
  --recovery-point-arn $DR_RP \
  --iam-role-arn arn:aws:iam::$ACCOUNT_ID:role/dr-backup-role \
  --metadata TargetTableName=orders-dr \
  --region $DR --query RestoreJobId --output text)

until [ "$(aws backup describe-restore-job --restore-job-id $RESTORE_JOB \
  --query Status --output text --region $DR)" = "COMPLETED" ]; do sleep 15; done

Compute the numbers:

RP_TIME=$(aws backup describe-recovery-point --backup-vault-name dr-vault \
  --recovery-point-arn $DR_RP --query CreationDate --output text --region $DR)
DONE=$(aws backup describe-restore-job --restore-job-id $RESTORE_JOB \
  --query CompletionDate --output text --region $DR)

python3 - "$T0" "$RP_TIME" "$DONE" <<'PY'
import sys, datetime as dt
p = lambda s: dt.datetime.fromisoformat(s.replace("Z", "+00:00"))
t0, rp, done = map(p, sys.argv[1:4])
print(f"Measured RPO: {(t0 - rp).total_seconds()/60:.1f} minutes of data loss exposure")
print(f"Measured RTO: {(done - t0).total_seconds()/60:.1f} minutes to a live table in the DR region")
PY

Verify it works

Confirm the restored table holds the seeded record, using only the DR region:

aws dynamodb get-item --table-name orders-dr \
  --key '{"id":{"S":"order-1001"}}' --region $DR

Expected output:

{
    "Item": {
        "id": {"S": "order-1001"},
        "total": {"N": "49.99"}
    }
}

The timestamp math should print something like:

Measured RPO: 42.3 minutes of data loss exposure
Measured RTO: 11.8 minutes to a live table in the DR region

Confirm the fault was real while you restored: aws fis get-experiment --id $EXP_ID --region $PRIMARY --query experiment.state shows "status": "running" during the window, and aws ec2 describe-network-acls --filters Name=tag:managedByFIS,Values=true --region $PRIMARY lists the cloned ACLs FIS swapped in. When you're done, end the fault early instead of waiting out the 30 minutes:

aws fis stop-experiment --id $EXP_ID --region $PRIMARY

If the measured RPO offends you, that's the point of the exercise: your worst-case RPO with an hourly schedule is an hour plus copy time, and now you have the number to prove it.

Troubleshooting

Feature is not supported for provided resource type on start-copy-job. Advanced DynamoDB backup wasn't enabled when the recovery point was created. Recovery points whose ARN starts with arn:aws:dynamodb: are DynamoDB-managed and can never be copied cross-region. Run the update-region-settings command from step 2, take a fresh backup, and copy that one; its ARN will start with arn:aws:backup:.

User: arn:aws:iam::...:user/you is not authorized to perform: iam:PassRole on resource: arn:aws:iam::...:role/dr-backup-role. Your CLI principal can't hand the service role to AWS Backup. Attach a policy granting iam:PassRole on dr-backup-role (and dr-fis-role) to whatever identity runs these commands.

FIS experiment goes straight to failed. Read the reason: aws fis get-experiment --id $EXP_ID --region $PRIMARY --query experiment.state. A target-resolution failure means $SUBNET_ARNS was empty or wrong; check that describe-subnets actually returned ARNs (no default VPC means no default-for-az subnets). An assume-role failure means the trust policy doesn't allow fis.amazonaws.com.

Restore job fails because the table exists. DynamoDB restores must target a table name that doesn't exist yet in that region. If a previous attempt created orders-dr, delete it (aws dynamodb delete-table --table-name orders-dr --region $DR) or pick a new TargetTableName.

Next steps

Tear down when finished: delete both tables, the backup selection and plan, the recovery points, both vaults, the FIS template, and the two IAM roles. Recovery points must be deleted before a vault can be.

From here, make the drill match production. Target your real app subnets instead of the default VPC and watch what your service actually does when DynamoDB disappears. Add a CloudWatch alarm stop condition so a game day can't hurt real users. Tighten RPO by moving to continuous backup with point-in-time restore where the resource type supports it, and cut RTO by scripting the restore path you just ran manually. FIS also ships a prebuilt Cross-Region: Connectivity scenario that isolates an entire region (VPC traffic, S3 replication, DynamoDB global tables) when you're ready to test more than one endpoint. Finally, put the whole thing on a schedule: a DR plan you last tested a year ago is a hypothesis, and hypotheses don't page anyone when they're wrong.