https://community.cloudera.com/t5/Archives-of-Support-Questions/hdfs-commands-not-working-from-StandBy-namenode-in-High/m-p/102544#M15063 <P>well, this was indeed unnecessary strugle... <span class="lia-unicode-emoji" title=":disappointed_face:">😞</span></P><P>I found the root cause in the logs of the now active namenode ("java.io.IOException: Cannot run program "/etc/hadoop/conf/rack-topology.sh")</P><P>when I enabled the High Availability in the cluster as a part of the exercises required by the admin course I simply forgot to provide the new namenode with the rack-topology.sh script (changing the topology was another prior exercise).</P><P>now all hdfs commands can be run from both namenodes</P> Thu, 14 Jan 2016 20:17:21 GMT jarourbtb 2016-01-14T20:17:21Z https://community.cloudera.com/t5/Archives-of-Support-Questions/hdfs-commands-not-working-from-StandBy-namenode-in-High/m-p/102543#M15062 <P>Dear fellow hadoopers,</P><P>I use HDPv2.2 sandbox for learning purposes. Some time ago I did all lab exercises for the Hortonworks Admin Course ("HDP Operations: Install and Manage with Apache Ambari"). Yesterday I turned the machine on and wanted to delve deeper into the administration. I noticed a strange thing - I can run "hdfs dfs -cat SOMEFILE" command on any file on the hdfs system from any of my nodes EXCEPT from the StandBy namenode.</P><P>The debugging ("HADOOP_ROOT_LOGGER='DEBUG,console' hdfs dfs -cat merged.txt") showed the following problem:</P><P>"INFO retry.RetryInvocationHandler: Exception while invoking getFileInfo of class ClientNamenodeProtocolTranslatorPB over node1/172.17.0.2:8020. Trying to fail over immediately. org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby"</P><P>On the StandBy namenode (node1) I can only run "hdfs dfs -ls" and "hdfs dfs -put" commands everything else fails with NullPointerException.</P><P>The checked the service states of both standby (node1, standby) and active (node4, active) namenodes using (hdfs haadmin -getServiceState).</P><P>Does anyone have an idea of what could be wrong? Why in case of my standby node the hdfs commands are not sent to the active node?</P><P>Thank you very much</P><P>jaro</P> Thu, 14 Jan 2016 19:18:53 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/hdfs-commands-not-working-from-StandBy-namenode-in-High/m-p/102543#M15062 jarourbtb 2016-01-14T19:18:53Z https://community.cloudera.com/t5/Archives-of-Support-Questions/hdfs-commands-not-working-from-StandBy-namenode-in-High/m-p/102544#M15063 <P>well, this was indeed unnecessary strugle... <span class="lia-unicode-emoji" title=":disappointed_face:">😞</span></P><P>I found the root cause in the logs of the now active namenode ("java.io.IOException: Cannot run program "/etc/hadoop/conf/rack-topology.sh")</P><P>when I enabled the High Availability in the cluster as a part of the exercises required by the admin course I simply forgot to provide the new namenode with the rack-topology.sh script (changing the topology was another prior exercise).</P><P>now all hdfs commands can be run from both namenodes</P> Thu, 14 Jan 2016 20:17:21 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/hdfs-commands-not-working-from-StandBy-namenode-in-High/m-p/102544#M15063 jarourbtb 2016-01-14T20:17:21Z