Support Questions
Find answers, ask questions, and share your expertise

Does oozie using distcp via webhdfs requires updated in core-site.xml?

New Contributor

Hi,

I have two clusters which are independent and need to do distcp.

ClusterA (has webhdfs enabled)

ClusterB(which needs to fetch data)

on ClusterB, if I can access data using Hadoop cli,

  1. hadoop fs -ls hdfs://clusterA/user/sanjeev/files/yyyy/mm/dd/hh
  2. hadoop fs -ls webhdfs://clusterA/user/sanjeev/files/files/yyyy/mm/dd/hh
I can do distcp via cli on clusterB,
but when I schedule oozie, which uses oozie-distcp-action, I am getting

Error: E0803 : E0803: IO error, Unauthorized connection for super-user: oozie from IP 192.100.12.1

does clusterA, requires my oozie hostname/ip to be present in core-site.xml?

or is there any other better way when webhdfs is enabled on clusterA, to do distcp to clusterB?

1 REPLY 1

Re: Does oozie using distcp via webhdfs requires updated in core-site.xml?

Super Mentor

@Sanjeev Tripurari

The error indicates that you might have not set the "" property properly in your HDFS configs.

 Unauthorized connection for super-user: oozie from IP 192.100.12.1

.

You can check if you have the following entries there or not? In Ambari Web, browse to Services > HDFS > Configs.

Under the Advanced tab, navigate to the Custom core-site section.

Click Add Property… to add the following custom properties: (In the hadoop.proxyuser.oozie.hosts either add comma separated list of addresses or *)
hadoop.proxyuser.oozie.hosts=*
hadoop.proxyuser.oozie.groups=*

.