Cloning S3 Buckets with aws-cli

The first step is to create your new S3 bucket (Copying the permissions from the source bucket to ensure they are the same).

Next, configure your linux server with the awscli:

sudo apt-get install awscli
aws configure

Enter your ACCESS_ID and SECRET_ACCESS_ID from your AWS IAM account and the default region of eu-west-1 (Ireland) – Adjust as appropriate.

Once you have awscli configured, run the below command to run a dry run of the sync the data between the two buckets. This will print out a list of s3 commands that will be run to complete the copy:

aws s3 sync s3://source-bucket s3://destination-bucket --dryrun && echo "Completed at: $(date +%F-%H:%M)"

If you are happy with the above – Run the same command without –dryrun to initiate the copy.

Uploading to an S3 bucket:

Uploading can be done in the same way as the s3 copy above (Replacing the first location with a local directory – Eg:

aws s3 sync /local/directory/ s3://destination-bucket --dryrun && echo "Completed at: $(date +%F-%H:%M)"

Again, once you are happy with the above – Remove –dryrun¬†to run the upload.

Any comments or questions? Get in touch here or Email me at [email protected]