How to Achieve AWS S3 Copy from One Bucket

Can you copy data from one S3 bucket to another? The answer is absolutely yes. You can use several methods to realize copy, and this article provides you with some ways to help you successfully copy S3 object.

By @Amelia Last Updated December 10, 2024
 

AWS S3 copy files and folders between two buckets

I have been on the lookout for a tool to help me copy content of an AWS S3 bucket into a second AWS S3 bucket without downloading the content first to the local file system.

- Question from Stack Overflow

Amazon S3 is a cloud storage solution. The most common task when using S3, either between buckets within the same AWS account or between different AWS accounts, is to copy the S3 object. This article will provide you with a method for cross-account and same-account AWS S3 copy from one bucket.

How the AWS S3 Copy Bucket to Another Bucket

When you choose to copy objects between S3 buckets, you can achieve this in the same account or a different account, the following section will provide several methods to guide you through both scenarios.

AWS S3 Copy from one Bucket in the Same Account

1. Using the AWS Management Console

Step 1. Sign in to the AWS Management Console, navigate to Buckets, and select General Purpose Buckets.

Step 2. Select the bucket you want to copy, on the Action dropdown menu, choose Copy.

Step 3. Navigate to the destination tab, click Action, and select Paste.

2. Using AWS CLI

Step 1. Open your terminal, and run the copy command.

aws s3 cp s3://source-bucket-name/object-key s3://destination-bucket-name/object-key

🎈Note: Remember to replace source-bucket-name with the name of your source bucket, object-key with the key of the object you want to copy and destination-bucket-name with the name of your destination bucket.

Step 2. After copying, check the copy by inspecting the target bucket in the AWS Management Console or using the CLI to list the objects in the target bucket.

aws s3 ls s3://my-destination-bucket/

3. Using SDKs

Here is an example using Python with Boto3 to copy one bucket.

import boto3   s3 = boto3.client('s3')   source_bucket = 'my-source-bucket' destination_bucket = 'my-destination-bucket' object_key = 'my-object.txt'   copy_source = { 'Bucket': source_bucket, 'Key': object_key }   s3.copy_object(CopySource=copy_source, Bucket=destination_bucket, Key=object_key) print(f'Copied {object_key} from {source_bucket} to {destination_bucket}.')

AWS S3 Copy from one Buckey in the Different Account

1. Granting Permission

The source bucket must have bucket policies allowing target accounts to access objects, so you must set bucket policies. Following is an example to add the source bucket:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::DESTINATION_ACCOUNT_ID:root" }, "Action": "s3:GetObject", "Resource": "arn:aws:s3:::source-bucket-name/*" } ] }

Also, make sure that the IAM user or role in the destination account has permission to perform the s3:CopyObject operation.

2. Using AWS CLI

Use the aws s3 cp command to specify the source bucket’s URL:

aws s3 cp s3://source-bucket-name/object-key s3://destination-bucket-name/object-key --source-region us-west-2

3.Using SDKs

Following is an example in Python to copy bucket:

import boto3   # Create a session with the source account credentials session = boto3.Session( aws_access_key_id='SOURCE_ACCOUNT_ACCESS_KEY', aws_secret_access_key='SOURCE_ACCOUNT_SECRET_KEY' )   s3_source = session.client('s3')   # Create a session with the destination account credentials session_dest = boto3.Session( aws_access_key_id='DESTINATION_ACCOUNT_ACCESS_KEY', aws_secret_access_key='DESTINATION_ACCOUNT_SECRET_KEY' )   s3_dest = session_dest.client('s3')   source_bucket = 'source-bucket-name' destination_bucket = 'destination-bucket-name' object_key = 'my-object.txt'   copy_source = { 'Bucket': source_bucket, 'Key': object_key }   # Copy the object from the source bucket to the destination bucket s3_dest.copy_object(CopySource=copy_source, Bucket=destination_bucket, Key=object_key) print(f'Copied {object_key} from {source_bucket} to {destination_bucket}.')

Securely Back up Your Data to Amazon S3

During the copy process, some situations may lead to data loss, so for your data security, you can choose a backup software to do so. AOMEI Cyber Backup is a professional software that can provide you witha comprehensive function to protect critical data across various environments, including physical and virtual infrastructures.

Below are reasons why you choose AOMEI Cyber Backup:

Versatile Backup Options: It supports full, incremental, and differential backups, catering to different data protection needs. User-Friendly Interface: It has an intuitive interface that allows users of all skill levels to easily navigate and perform backups. Flexible Scheduling: It allows users to set up automated backup schedules, ensuring regular data protection without manual intervention. Compatible with Amazon S3: It can seamlessly integrate with Amazon S3 andlets you take advantage of AWS’ powerful and scalable cloud storage. Easy Restoration: It provides straightforward data restoration options, allowing for quick recovery in case of data loss corruption.

Download FreewareCentralized Backup Solution
Secure Download

Follow these steps to begin AOMEI Cyber Backup with Amazon S3:

Step 1: Navigate to Target Storage > Amazon S3 > + Add Target to open the Add Target page. Then enter the following information: Username, Password, and Bucket, and click Confirm.

Step 2. Navigate to Backup Task > Create New Task to start archiving your important data to Amazon S3. For example, select File Backup and choose files or folders for backup.

Step 3. Check Archiving backup versions to Amazon S3 and click Select to choose the added Amazon S3.

Step 4. Schedule backup tasks to run daily, weekly, or monthly, and choose backup retention policies to automatically delete old backups.

Step 5. Click Start Backup to begin the backup process. It will first create a backup locally or on the NAS and then upload the backup to Amazon S3. According to the 3-2-1 backup rule, this ensures the security of critical data and business continuity.

Conclusion

AWS S3 copy from one bucket is a simple process, and you can implement same-account or cross-account S3 object copying in a variety of ways, such as through the AWS Management Console, CLI, or SDK, etc. However, during the copying process, be sure you have the necessary permissions and consider the transfer costs associated with copying large amounts of data.