I need to copy hdfs data from cluster to s3.
If I use s3a, I get:
WARN s3a.S3AFileSystem: Client: Amazon S3 error 400: 400 Bad Request; Bad Request (retryable)
If I instead try with S3, it says:
The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
My bucket is in Asia/Mumbai region and I tried creating one in US east. Nothing changed.
A similar issue fix by applying attached patch in this link. It 'may' help you