When the local NameNode is healthy, the ZKFC holds a session open in ZooKeeper. If the local NameNode is active, it also holds a special “lock” znode. This lock uses ZooKeeper’s support for “ephemeral” nodes; if the session expires, the lock node will be automatically deleted. Ephemeral znodes - These znodes exists as long as the session that created the znode is active. When the session ends the znode is deleted. Zookeeper has a concept of Watches which can be set on znodes to track them for any changes. Check the link how Watches work in zookeeper - https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkWatches ... View more

We cannot set specfic configuration to only handle replication related jobs with separate NIC in HDFS. However, we can setup a multi-homed environment for datanodes for all the traffic for specific datanodes. Kindly check below documentation for the same - https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html ... View more

@George-Megre  The Master nodes firstly are not meant for launching tasks or interaction.  Edge or gateway nodes are used to run client applications and cluster administration tools. Setup of Edge/gateway node is similar to any Hadoop node except no Hadoop cluster services runs on the GW/Edge nodes they are mere entry points and connection gateway to the Master Components like HDFS (NN) HBASE etc provide you have installed the client libraries.   In your case, I am sure you have the Hbase client/gateway roles on the 3 nodes and not on the master noded. The HBase client role gives the connectivity to Hbase but again I don't see why you would like to connect /initiate the HBase shell from the master node? Geoffrey ... View more

Hello @yagoaparecidoti    Thanks for using Cloudera Community. Based on the Post, You encountered "Master Is Initializing" & ended up fixing the same using Command found on Internet. You ended up with few Tables in Broken State, requiring you to delete & recreate them.   For documenting the Post, the Issue observed is tracked via [1]. In the HMaster Logs, We should see HBase:Meta & HBase:Namespace Region, which aren't being assigned. Sample tracing shared via [1]. Once the same happens, Using HBCK2 Jar to assign the Region being reported as Unassigned is required. The Command to be used via HBCK2 Jar to assign the Region is shared via [1] as well.    The HBCK2 Command has other Options, which (If performed without any oversight) may cause issues with HBase Table availability. The "Master Is Initializing" Error Fix as shared via [1] shouldn't cause any Table to be in Broken State & without knowing the explicit details into the Broken State, It's harder to confirm if Delete-Recreate was the ONLY Way Or, We could have restored the Table in other ways.   Having said that, the Post covers "Master Is Initializing" Error & the same has been addressed. As such, We shall mark the Post as Closed. Feel free to share your observation with the Tables in Broken State in a New Post, if you wish to engage for the Community observation & feedback.    Regards, Smarak   [1] https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2#master-startup-cannot-progress-in-holding-pattern-until-region-onlined   ... View more