Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

distcp from cluster to s3

Expert Contributor

I need to copy hdfs data from cluster to s3.

 

If I use s3a, I get:

 

 

 WARN s3a.S3AFileSystem: Client: Amazon S3 error 400: 400 Bad Request; Bad Request (retryable)

 

If I instead try with S3, it says:

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

 

My bucket is in Asia/Mumbai region and I tried creating one in US east. Nothing changed.

 

1 REPLY 1

Champion

@sim6

 

A similar issue fix by applying attached patch in this link. It 'may' help you

 

https://issues.apache.org/jira/browse/HADOOP-13324

 

 

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.