Member since
06-06-2016
2
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
9459 | 06-07-2016 02:09 AM |
06-07-2016
02:09 AM
2 Kudos
Hi Harsh Thanks for your reply. Yarn gateway was missing and adding them fixed the issue.
... View more
06-06-2016
09:16 AM
Hi all, We have just install a Kerberized cluster. I would like to copy data from the former unsecured cluster (opendatarec) to the secured one (opendata) using distcp. I run the following command on the secured cluster : [hdfs@opendatafr01f ~]$ hadoop distcp webhdfs://opendatarecm01f.bi3.local:50070/toto webhdfs://opendatam01f.bi3.local:50070/tmp/ and get the following error : mp/ 16/06/06 17:46:55 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsize', sourceFileListing=null, sourcePaths=[webhdfs://opendatarecm01f.bi3.local:50070/toto], targetPath=webhdfs://opendatam01f.bi3.local:50070/tmp, targetPathExists=true, preserveRawXattrs=false, filtersFile='null'} 16/06/06 17:46:55 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 16/06/06 17:46:55 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 16/06/06 17:46:55 ERROR tools.DistCp: Exception encountered java.io.IOException: Can't get Master Kerberos principal for use as renewer at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:133) at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100) at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80) at org.apache.hadoop.tools.SimpleCopyListing.validatePaths(SimpleCopyListing.java:164) at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:85) at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:90) at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86) at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:372) at org.apache.hadoop.tools.DistCp.execute(DistCp.java:172) at org.apache.hadoop.tools.DistCp.run(DistCp.java:123) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.tools.DistCp.main(DistCp.java:436) Any suggestion ? Thanks
... View more
Labels:
- Labels:
-
Apache Hadoop
-
HDFS
-
Kerberos
-
MapReduce
-
Security