Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Running distcp between two cluster: One Kerberized and the other is not

Solved Go to solution
Highlighted

Re: Running distcp between two cluster: One Kerberized and the other is not

New Contributor
  1. getting below error after running the command "hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true hdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo"
  2. java.io.EOFException:End of FileException between local host is***; destination host is:***;
  3. please suggest

Re: Running distcp between two cluster: One Kerberized and the other is not

New Contributor
getting below error after running the command "hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true hdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo"

java.io.EOFException: End of File Exception between local host is *** ; destination host is:***;


please suggest

Re: Running distcp between two cluster: One Kerberized and the other is not

New Contributor

I have found that using webhdfs instead of hdfs for the nonsecure host gets around this error, e.g.,

"hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true webhdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo"

Re: Running distcp between two cluster: One Kerberized and the other is not

New Contributor

can you try this

hadoop distcp –D ipc.client.falback-tosimple-auth-allowed=true webhdfs://insecureCluster webhdfs://secureCluster

Re: Running distcp between two cluster: One Kerberized and the other is not

New Contributor

We had similar issue.

Most likely you have following property in /etc/hadoop/conf/hdfs-site.xml:

<property>
  <name>dfs.namenode.acls.enabled</name>
  <value>true</value>
  </property

Remove this property or set it to "false". It should help.

Re: Running distcp between two cluster: One Kerberized and the other is not

New Contributor

Am doing distcp from insecure to secure hadop cluster and am getting error "SIMPLE authentication isnot enabled". Can any one suggest.

hdfs@master02:~> hadoop distcp -Dipc.client.fallback-to-simple-auth-allowed=true hdfs://HDP23:8020/test01.txt hdfs://HDP24:8020/

17/04/0500:09:28 ERROR tools.DistCp:Invalid arguments:org.apache.hadoop.security.AccessControlException: SIMPLE authentication isnot enabled.Available:[TOKEN, KERBEROS]

Re: Running distcp between two cluster: One Kerberized and the other is not

New Contributor

LiteBlue is the U.S. Government website this is intended for authorized use only by Postal Service employees