Member since
11-28-2016
8
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
8399 | 11-29-2016 03:27 AM |
11-29-2016
03:27 AM
1 Kudo
SOLUTION FOUND: since we copy the data from unsecure cluster to secure cluster we need to set the property "ipc.client.fallback-to-simple-auth-allowed" to true in the secured cluster. We added the host entries of the unsecured cluster in the secured cluster. Now everything is working fine!
... View more
11-29-2016
01:18 AM
Thanks for the alternatives Wes Floyd, It works one way again. WEB HDFS : Yes WEBHDFS works from cluster A again. From Cluster B we use a numeric UserID because of which webhdfs call is failing, I see a patch for this anyways https://issues.apache.org/jira/browse/HDFS-4983. Is this patch not available as part HDP 2.4 ! Other cases: We tried to add -fs option same result Most weird part is hadoop fs -ls hdfs://a.a.a.a:8020, it still not throwing error it still gives the values of the cluster B. Extra Cluster Info: Cluster B has WanDisco installed, is this setup causing any issue. I hope we use WanDisco fusion client Jar instead of the hadoop Jars at the backend. Any pointers here! core_site.xml Entries: <property>
<name>fs.hdfs.impl</name>
<value>com.wandisco.fs.client.FusionHdfs</value>
</property> <property>
<name>fusion.handshakeToken.dir</name>
<value>##SOME DIR##</value>
</property> <property>
<name>fusion.ihc.http.policy</name>
<value>HTTP_ONLY</value>
</property> <property>
<name>fusion.ihc.ssl.enabled</name>
<value>true</value>
</property>
... View more
11-28-2016
06:11 AM
1 Kudo
More info. on the cluster: we have cluster A & B. cluster A - HDP 2.3 - not kerberised cluster B - HDP 2.4 - kerberised Action: we are trying to list the directories in cluster A by issuing the command from one of the client machine of cluster B. "hadoop fs -Dfs.defaultFS=<A namenode IP:port> -ls / " OR "hadoop fs -ls hdfs://<A namenode IP:port>/" we are able to list the directories of cluster B from cluster A client machine. Expectation Result: should list the directories from cluster A HDFS Actual Result: we are getting the list of files in the current B instead of cluster A. Reverse listing is working fine! Request: Please provide some pointers on what could have been the issue to enable -D option or what actually is stopping us to list the directories in the remote cluster.
... View more
Labels:
- Labels:
-
Apache Hadoop