I am using this https://community.hortonworks.com/articles/59161/using-hadoop-credential-api-to-store-aws-secrets.ht... as a reference, but unable to successfully distcp or even -ls.
[root@test232 conf]# hadoop distcp -Dhadoop.security.credential.provider.path=jceks://hdfs/aws/aws.jceks /nsswitch.conf s3a://kartik-test.s3-us-west-2.amazonaws.com
17/01/15 08:16:22 ERROR tools.DistCp: Invalid arguments:
java.io.IOException: Bucket kartik-test.s3-us-west-2.amazonaws.com does not exist
at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:229)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.hadoop.tools.DistCp.setTargetPathExists(DistCp.java:224)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:116)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:454)
Invalid arguments: Bucket kartik-test.s3-us-west-2.amazonaws.com does not exist
I am now reading this: https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html
but unable to understand what are the xml file names that need to be edited.
