Migrating S3 Buckets from AWS to GovCloud

Amazon S3 (Simple Storage Service) is a handy tool for online file storage that is often used when setting up VMs. While transferring data between availability zones in Amazon Web Services (AWS) is a trivial task, transferring S3 buckets from AWS to AWS GovCloud requires a greater challenge.

In order to start, you’ll need a few things first:

• Your AWS Access Key ID (for both AWS and AWS GovCloud)
• Your AWS Secret Access Key (for both AWS and AWS GovCloud)
• The region your S3 Bucket(s) are stored in

Finding your Access Key ID  and Secret Access Key has been well documented here.  

The first step is to configure the AWS CLI to allow us to sync all of our assets to our local machine.  Start by downloading and configuring the Amazon Web Services CLI. 

C:\Users\Alan\Documents\S3 - Assests>aws configure
AWS Access Key ID [****************ABCD]:
AWS Secret Access Key [****************EF12]:
Default region name [US Standard]: us-east-1
Default output format [None]:

Once your configured, you’ll want to sync all your access locally.  You’ll run the aws s3 sync command and select your S3 URL as the source and your local folder as your destination.  Bucket URL’s are typically formed using the following formats (http(s)://s3.amazonaws.com/<bucket_name>/<object> or http(s)://<bucket_name>.s3.amazonaws.com/<object>).

C:\Users\Alan\Documents\S3 - Assests>aws s3 sync s3://assets.myURL.com .
download: s3://assets.myURL.com/media/docs-page/myPDF.pdf to media\docs-page\myPDF.pdf

Now that you have all the assets downloaded locally from your AWS S3 bucket, you’ll need to reconfigure your AWS CLI for GovCloud to sync all your assets to the GovCloud S3.  You’ll reconfigure the CLI with your GovCloud Access ID and Key.  The zone is tricky because it’s not well documented but you’ll use us-gov-west-1.  Follow up configuration by performing the sync from local to your S3 URL in GovCloud.

C:\Users\Alan\Documents\S3 - Assests>aws configure
AWS Access Key ID [****************DCBA]:
AWS Secret Access Key [****************12FE]:
Default region name [us-gov-west]: us-gov-west-1
Default output format [None]:
C:\Users\Alan\Documents\S3 - Assests>aws s3 sync . s3://assets.newURL.com

upload: media\docs-page\myPDF.pdf to s3://assets.newURL.com/myPDF.pdf

Lastly, verify your directory structure looks correct and all assets were properly uploaded and you’re good to go!  If you use multiple availability zones, you’ll want to perform the same steps for each s3 bucket in each zone.