Code Destine

In this tutorial, we will learn about how to get a list of objects stored in S3 bucket using java language.

Project Setup

Create a simple maven project in your favorite IDE and add below mentioned dependency in your pom.xml file.

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk-s3</artifactId>
  <version>1.11.533</version>
</dependency>

For latest version of aws library, check this page.

S3 Connection

Create an object of AmazonS3 ( com.amazonaws.services.s3.AmazonS3 ) class for sending a client request to S3. To get instance of this class, we will use AmazonS3ClientBuilder builder class. It requires three important parameters :- 

1. Region :- It is a region where S3 table will be stored.
2. ACCESS_KEY :- It is a access key for using S3. You can generate this key, using aws management console.
3. SECRET_KEY :- It is a secret key of above mentioned access key.

Here is a code example :-

AmazonS3 s3 = AmazonS3ClientBuilder.standard()
				.withRegion(Regions.AP_SOUTH_1)
				.withCredentials(new AWSStaticCredentialsProvider
                                                    (new BasicAWSCredentials("ACCESS_KEY","SECRET_KEY")))
				.build();

List Objects

An AmazonS3.listObjects method returns a list of summary information about the objects stored in the specified bucket or prefix. The list of objects is always returned in lexicographic (alphabetical) order.

Amazon S3 buckets can contain an unlimited number of objects and requesting a complete list of objects can be time-consuming task. So to manage such a large result set, Amazon S3 uses pagination to split them into multiple responses. 

The ObjectListing.isTruncated() method can be used to check if, all the objects are returned or not. If not, then additional calls can be made using AmazonS3.listNextBatchOfObjects( objectList ) to get more results.

Example

import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.S3ObjectSummary;

public class ListObjects {

	public static void main(String[] args) {

		/* Create S3 Client Object */
		AmazonS3 s3 = AmazonS3ClientBuilder
				.standard()
				.withRegion(Regions.AP_SOUTH_1)
				.withCredentials(new AWSStaticCredentialsProvider(
                                      new BasicAWSCredentials("ACCESS_KEY","SECRET_KEY")))
				.build();
		
		try {	
			/* Get first batch of objects in a given bucket */
			ObjectListing objects = s3.listObjects("lb-aws-learning");
			
			/* Recursively get all the objects inside given bucket */
			if(objects != null && objects.getObjectSummaries() != null) {				
				while (true) {					
					for(S3ObjectSummary summary : objects.getObjectSummaries()) {
						System.out.println("Object Id :-" + summary.getKey());
					}
					
					if (objects.isTruncated()) {
						objects = s3.listNextBatchOfObjects(objects);
	                } else {
	                    break;
	                }					
				}
			}
            
        } catch (AmazonServiceException e) {
        	
            System.out.println(e.getErrorMessage());
            
        } finally {
        	
        	if(s3 != null) {
        		s3.shutdown();
        	}        	
        }
	}
}

Similar Post

1. How to use aws s3 sync command
2. How to copy object from one S3 bucket to another using Java
3. How to use aws s3 sync command
4. How to use aws s3 ls command

References :-

1. ListObjectsRequest Java Docs
2. AmazonS3 Java Docs

That’s all for how to get a list of objects stored in S3 bucket using java language. If you liked it, please share your thoughts in comments section and share it with others too.