We have a need to access external S3 buckets with "requester pays" enabled so that the bucket holder only pays for hosting the data but the cost of accessing it is paid for by the requester. Is there a way to enable this in Ambari? I'm specifically looking for a way to add the appropriate s3a property to enable this. As per https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html documentation from AWS, for EMR, it can be configured in the emrfs-site.xml with "fs.s3.useRequesterPaysHeader": "true" I am looking to see if there is a similar way to do this within Ambari. I also noticed that there is patch that was released for Apache Hadoop (https://issues.apache.org/jira/browse/HADOOP-14661) but I wanted to check how this patch can be applied or if Ambari has already incorporated this in a future release. If Ambari does not have this patch included, is there a way to apply this patch manually until it is officially available? Thanks in advance!
... View more