Posts: 66
Registered: ‎05-19-2016

distcp from cluster to s3

I need to copy hdfs data from cluster to s3.


If I use s3a, I get:



 WARN s3a.S3AFileSystem: Client: Amazon S3 error 400: 400 Bad Request; Bad Request (retryable)


If I instead try with S3, it says:

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.


My bucket is in Asia/Mumbai region and I tried creating one in US east. Nothing changed.


Posts: 418
Topics: 13
Kudos: 67
Solutions: 39
Registered: ‎09-02-2016

Re: distcp from cluster to s3



A similar issue fix by applying attached patch in this link. It 'may' help you



New solutions