Support Questions

Find answers, ask questions, and share your expertise

Distcp between Secured HA cluster to Non Secured HA cluster


We have set up two clusters, one non secure HA cluster and other one kerberos enabled HA cluster. We want to copy the data from Non secure HA to secure HA cluster using distcp. We run the following command from Kerberos enabled HA cluster:

# hadoop distcp -Dipc.client.fallback-to-simple-auth-allowed=true hdfs://non-secure-cluster-ID/user/mazin/test-file hdfs://secure-cluster-ID/user/mazin/

we get following error:

On client datanode which is in secured cluster - hadoop.log End of File Exception between local host is: ""; destination host is: "":8020; :; For more details see:

On non-secure namenode2001 I see following error:

2017-11-10 03:28:36,469 INFO org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8020: readAndProcess from client threw exception [java.lang.IllegalArgumentException: Illegal principal name mazin@EXAMPLE.COM:$NoMatchingRule: No rules applied to mazin@EXAMPLE.COM] java.lang.IllegalArgumentException: Illegal principal name mazin@EXAMPLE.COM:$NoMatchingRule: No rules applied to mazin@EXAMPLE.COM at<init>( at<init>( at at at org.apache.hadoop.util.ProtoUtil.getUgi( at org.apache.hadoop.util.ProtoUtil.getUgi( at org.apache.hadoop.ipc.Server$Connection.processConnectionContext( at org.apache.hadoop.ipc.Server$Connection.processRpcOutOfBandRequest( at org.apache.hadoop.ipc.Server$Connection.processOneRpc( at org.apache.hadoop.ipc.Server$Connection.readAndProcess( at org.apache.hadoop.ipc.Server$Listener.doRead( at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop( at org.apache.hadoop.ipc.Server$Listener$ Caused by:$NoMatchingRule: No rules applied to mazin@EXAMPLE.COM at at<init>( ... 12 more

Above error hints on entry in core-site.xml, since this is non secure cluster why do we need auth_to_local_entry setting.

Note: we want to use hdfs:// to copy the file and not webhdfs://

Thanks in advance.



This issue got fixed after the changed the from "DEFAULT" to "" on non secured core-site.xml. = ""