question Re: Running distcp between two cluster: One Kerberized and the other is not in Support Questions

Running distcp between two cluster: One Kerberized and the other is not

dstreev — Sat, 26 Sep 2015 00:58:11 GMT

hadoop distcp -i -log /tmp/ hdfs://xxx:8020/apps/yyyy hdfs://xxx_cid/tmp/

In this case the "xxx" is the "un-secure" cluster, while "xxx_cid" in the secure cluster.

We are launching the job from the Kerberos cluster, with the appropriate kinit for the user and getting the following error:

java.io.IOException: Failed on local exception: java.io.IOException: Server asks us to fall back to SIMPLE auth, but this client is configured to only allow secure connections.; Host Details : local host is: "xxx/10.x.x.x"; destination host is: "xxx":8020;

...

Caused by: java.io.IOException: Server asks us to fall back to SIMPLE auth, but this client is configured to only allow secure connections.

I thought by launching the job from the secure cluster, that we could avoid any access issues. But it appears that the processes are kicked off from the "source" cluster. In this case, that's the insecure cluster.

Idea's on getting around this?

Re: Running distcp between two cluster: One Kerberized and the other is not

stevel — Sat, 26 Sep 2015 17:28:37 GMT

sounds like the distcp process is running secure, but is configured to not like simple connections.

try setting the config option

ipc.client.fallback-to-simple-auth-allowed=true

Re: Running distcp between two cluster: One Kerberized and the other is not

pardeep_kumar — Tue, 29 Sep 2015 17:20:59 GMT

@dstreever@hortonworks.com To use Distcp for copying between a secure cluster and an insecure one, add the following to the HDFS core-default.xml, by using Ambari.

<property>
  <name>ipc.client.fallback-to-simple-auth-allowed</name>
  <value>true</value> 
</property>

Re: Running distcp between two cluster: One Kerberized and the other is not

nsabharwal — Thu, 05 Nov 2015 00:09:06 GMT

@Pardeep Nice find! Link

When copying data from a secure cluster to a secure cluster, the following configuration setting is required in the core-site.xml file:

<property>
    <name>hadoop.security.auth_to_local</name>
    <value></value>
    <description>Maps kerberos principals to local user names</description>
</property>

Re: Running distcp between two cluster: One Kerberized and the other is not

cnauroth — Thu, 19 Nov 2015 06:25:21 GMT

I recommend not setting this in core-site.xml, and instead setting it on the command line invocation specifically for the DistCp command that needs to communicate with the unsecured cluster. Setting it in core-site.xml means that all RPC connections for any application are eligible for fallback to simple authentication. This potentially expands the attack surface for man-in-the-middle attacks.

Here is an example of overriding the setting on the command line while running DistCp:

hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true hdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo

The command must be run while logged into the secured cluster, not the unsecured cluster.

Re: Running distcp between two cluster: One Kerberized and the other is not

nsabharwal — Sat, 21 Nov 2015 22:46:29 GMT

@Chris Nauroth Thanks for sharing this. Could you update the answer with more details? I believe this is the best answer if you can add more details.

Re: Running distcp between two cluster: One Kerberized and the other is not

cnauroth — Tue, 24 Nov 2015 05:49:42 GMT

@Neeraj Sabharwal, thank you. I updated the answer to show an example of overriding the property from the DistCp command line.

Re: Running distcp between two cluster: One Kerberized and the other is not

nsabharwal — Tue, 24 Nov 2015 07:37:11 GMT

Thanks @Chris Nauroth

Re: Running distcp between two cluster: One Kerberized and the other is not

hari_prsd14 — Wed, 16 Mar 2016 23:35:16 GMT

getting below error after running the command "hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true hdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo"
java.io.EOFException:End of FileException between local host is***; destination host is:***;
please suggest

Re: Running distcp between two cluster: One Kerberized and the other is not

hari_prsd14 — Wed, 16 Mar 2016 23:36:45 GMT

getting below error after running the command "hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true hdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo"
java.io.EOFException:End of FileException between local host is***; destination host is:***;
please suggest

Re: Running distcp between two cluster: One Kerberized and the other is not

hari_prsd14 — Wed, 16 Mar 2016 23:44:46 GMT

getting below error after running the command "hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true hdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo"

java.io.EOFException: End of File Exception between local host is *** ; destination host is:***;


please suggest

Re: Running distcp between two cluster: One Kerberized and the other is not

kshaffer_tpgs — Fri, 13 May 2016 21:09:52 GMT

I have found that using webhdfs instead of hdfs for the nonsecure host gets around this error, e.g.,

"hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true webhdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo"

Re: Running distcp between two cluster: One Kerberized and the other is not

saramakr — Wed, 25 May 2016 20:12:53 GMT

can you try this

hadoop distcp –D ipc.client.falback-tosimple-auth-allowed=true webhdfs://insecureCluster webhdfs://secureCluster

Re: Running distcp between two cluster: One Kerberized and the other is not

leonidzav — Wed, 08 Mar 2017 04:28:48 GMT

We had similar issue.

Most likely you have following property in /etc/hadoop/conf/hdfs-site.xml:

<property>
  <name>dfs.namenode.acls.enabled</name>
  <value>true</value>
  </property

Remove this property or set it to "false". It should help.

Re: Running distcp between two cluster: One Kerberized and the other is not

turing_nix — Wed, 05 Apr 2017 02:03:10 GMT

Am doing distcp from insecure to secure hadop cluster and am getting error "SIMPLE authentication isnot enabled". Can any one suggest.

hdfs@master02:~> hadoop distcp -Dipc.client.fallback-to-simple-auth-allowed=true hdfs://HDP23:8020/test01.txt hdfs://HDP24:8020/

17/04/0500:09:28 ERROR tools.DistCp:Invalid arguments:org.apache.hadoop.security.AccessControlException: SIMPLE authentication isnot enabled.Available:[TOKEN, KERBEROS]

Re: Running distcp between two cluster: One Kerberized and the other is not

jkiran2020 — Wed, 28 Mar 2018 06:23:18 GMT

Adding this property in core-site.xml helped resolve the error.

Re: Running distcp between two cluster: One Kerberized and the other is not

badmanlive085 — Wed, 17 Apr 2019 13:21:55 GMT

LiteBlue is the U.S. Government website this is intended for authorized use only by Postal Service employees

Re: Running distcp between two cluster: One Kerberized and the other is not

irunker — Mon, 30 Sep 2024 14:41:51 GMT

非安全集群被阻止rpc通信，使用webhdfs协议，hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true webhdfs://nn1:50070/foo/bar hdfs://nn2:8020/bar/foo