Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

distcp from cluster to s3

distcp from cluster to s3

Expert Contributor

I need to copy hdfs data from cluster to s3.

 

If I use s3a, I get:

 

 

 WARN s3a.S3AFileSystem: Client: Amazon S3 error 400: 400 Bad Request; Bad Request (retryable)

 

If I instead try with S3, it says:

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

 

My bucket is in Asia/Mumbai region and I tried creating one in US east. Nothing changed.

 

1 REPLY 1

Re: distcp from cluster to s3

Champion

@sim6

 

A similar issue fix by applying attached patch in this link. It 'may' help you

 

https://issues.apache.org/jira/browse/HADOOP-13324