Code Destine

In this tutorial, we will learn about how to use aws s3 sync command using aws cli.

sync Command

The sync command is used to sync directories to S3 buckets or prefixes and vice versa. It recursively copies new and updated files from the source ( Directory or Bucket/Prefix ) to the destination ( Directory or Bucket/Prefix ). It only creates folders in the destination if they contain one or more files.

Optional Arguments

This command takes the following optional arguments :-

1. –dryrun :- It displays the operations that would be performed using the specified command without actually running them.
2. –quiet :- It does not display the operations performed from the specified command.
3. –include (string) :- It includes all files or objects in the command that match the specified pattern.
4. –exclude (string) :- It excludes all files or objects from the command that matches the specified pattern.
5. –storage-class (string) :- It is a type of storage class to be used for the object. Valid choices are STANDARD | REDUCED_REDUNDANCY | STANDARD_IA | ONEZONE_IA | INTELLIGENT_TIERING | GLACIER | DEEP_ARCHIVE. Defaults to ‘STANDARD‘
6. –source-region (string) :- When objects are transferred from an S3 bucket to an S3 bucket, this specifies the region of the source bucket. If –source-region is not specified, then the region of the source will be the same as the region of the destination bucket.
7. –page-size (integer) :- It returns the specified number of results in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out.
8. –size-only :- It makes the size of each key the only criteria used to decide whether to sync from source to destination.
9. –exact-timestamps :- During syncing from S3 to a local directory, the same-sized items will be ignored only when the timestamps match exactly. The default behavior is to ignore same-sized items unless the local directory version is newer than the S3 version.
10. –delete :- It deletes the files that exist in the destination but not in the source.

Syntax 

aws s3 sync <S3Uri> <LocalPath> or <S3Uri> <S3Uri> or <LocalPath> <S3Uri> [options]

Examples of aws s3 sync

Sync Local Directory => S3 Bucket/Prefix

The following sync command syncs objects inside a specified prefix or bucket to files in a local directory by uploading the local files to Amazon S3. A syncs operation from a local directory to S3 bucket occurs, only if one of the following conditions is met :-

1. Size :- If a size of the local file is different than the size of the s3 object.
2. Time :- If last modified time of the local file is greater than the last modified time of the s3 object.
3. Exist :- If a local file does not exist under the specified bucket or prefix.

In the example below, the user syncs the bucket lb-aws-learning to the local current directory.

Syntax

aws s3 sync . <S3Uri> 

Example

Sync S3 Bucket/Prefix => S3 Bucket/Prefix

The following sync command syncs objects under a specified prefix or bucket to objects under another specified prefix or bucket by copying S3 objects. A syncs operation from one S3 bucket to another S3 bucket occurs, only if one of the following conditions is met :-

1. Size :- If the size of the two Amazon S3 objects is different.
2. Time :- If last modified time of the source S3 object is greater than the last modified time of the destination S3 object.
3. Exist :- If an S3 object does not exist under the specified destination bucket or prefix.

In the example below, the user syncs the bucket lb-aws-learning to lb-aws-learning-1 bucket.

Syntax

aws s3 sync <S3Uri> <S3Uri> 

Example

Sync S3 Bucket/Prefix => Local Directory

The following sync command syncs files in a local directory to objects under a specified prefix or bucket by downloading S3 objects to the local directory. A syncs operation from an S3 bucket to local directory occurs, only if one of the following conditions is met :-

1. Size :- If the size of the S3 object is different than the size of the local file.
2. Time :- If last modified time of the S3 object is greater than the last modified time of the local file.
3. Exist :- If an S3 object does not exist in a local directory.

In the example below, the user syncs the local current directory to the bucket lb-aws-learning.

Syntax

aws s3 sync <S3Uri> . 

Example

Similar Post

1. How to upload an object to S3 bucket using Java
2. How to get a list of objects stored in S3 using Java
3. How to delete bucket in S3 using Java
4. How to create bucket in S3 using Java

References :-

1. S3 sync Command Docs

That’s all for how to use aws s3 sync command using aws cli. If you liked it, please share your thoughts in comments section and share it with others too.