Support Questions

Find answers, ask questions, and share your expertise

Distcp is failing in HA

avatar
Guru

When I am trying to do distcp in High Availability Cluster then it is failing with below error.

[s0998@test ~]$ hadoop distcp  hdfs://HDPINFHA/user/s0998/sampleTest.txt hdfs://HDPTSTHA/user/root/
16/02/29 06:32:38 ERROR tools.DistCp: Invalid arguments: 
java.lang.IllegalArgumentException: java.net.UnknownHostException: HDPTSTHA
	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:406)
	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
	at org.apache.hadoop.tools.DistCp.setTargetPathExists(DistCp.java:216)
	at org.apache.hadoop.tools.DistCp.run(DistCp.java:116)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.tools.DistCp.main(DistCp.java:430)
Caused by: java.net.UnknownHostException: HDPTSTHA

Though I have configured via below urls.

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0-Win/bk_HDP_RelNotes_Win/content/behav-change...

1 ACCEPTED SOLUTION

avatar
Master Mentor
11 REPLIES 11

avatar
Master Mentor

@Saurabh Kumar

You may want to open a support ticket.

Could you recheck all the steps as mentioned in the reply?

avatar
Guru

@Neeraj Sabharwal :

Thanks for your support, I found a issue actually there was an misconfiguration in hdfs-site.xml file.

I did not add target cluster HA properties to client hdfs-site.xml and because of that it was failing.

but now it is working fine.