Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

S3A to support Requester Pays Buckets

S3A to support Requester Pays Buckets

New Contributor

We have a need to access external S3 buckets with "requester pays" enabled so that the bucket holder only pays for hosting the data but the cost of accessing it is paid for by the requester.

 

Is there a way to enable this in Ambari? I'm specifically looking for a way to add the appropriate s3a property to enable this. As per https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html documentation from AWS, for EMR, it can be configured in the emrfs-site.xml with 

 

 

"fs.s3.useRequesterPaysHeader": "true"

 

 

 

I am looking to see if there is a similar way to do this within Ambari. I also noticed that there is patch that was released for Apache Hadoop (https://issues.apache.org/jira/browse/HADOOP-14661) but I wanted to check how this patch can be applied or if Ambari has already incorporated this in a future release.

 

If Ambari does not have this patch included, is there a way to apply this patch manually until it is officially available?

 

Thanks in advance!

Don't have an account?
Coming from Hortonworks? Activate your account here