distcp : Can't get Master Kerberos principal for use as renewer

[#1]

Hi all,

We have just install a Kerberized cluster.

I would like to copy data from the former unsecured cluster (opendatarec) to the secured one (opendata) using distcp.

I run the following command on the secured cluster :

[hdfs@opendatafr01f ~]$ hadoop distcp webhdfs://opendatarecm01f.bi3.local:50070/toto webhdfs://opendatam01f.bi3.local:50070/tmp/

and get the following error :

mp/
16/06/06 17:46:55 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsize', sourceFileListing=null, sourcePaths=[webhdfs://opendatarecm01f.bi3.local:50070/toto], targetPath=webhdfs://opendatam01f.bi3.local:50070/tmp, targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
16/06/06 17:46:55 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
16/06/06 17:46:55 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
16/06/06 17:46:55 ERROR tools.DistCp: Exception encountered
java.io.IOException: Can't get Master Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:133)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.tools.SimpleCopyListing.validatePaths(SimpleCopyListing.java:164)
at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:85)
at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:90)
at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:372)
at org.apache.hadoop.tools.DistCp.execute(DistCp.java:172)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:123)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:436)

Any suggestion ?

Thanks

[#2]

On the host where you are invoking this command, do you have proper client
configs (gateway roles if you use CM) properly deployed?

Please also look at our DistCp matrix to learn how/where to invoke the
DIstCp command depending on your situation:
http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_admin_distcp_data_cluster_migrate...

[#3 · Accepted solution]

Hi Harsh

Thanks for your reply.

Yarn gateway was missing and adding them fixed the issue.

[#4]

Glad to hear @AxelJ! Feel free to mark the thread resolved so its easier for others with similar problems to locate your solution.