Support Questions
Find answers, ask questions, and share your expertise

Failed to initialize High Availability state in ZooKeeper from Failover Controller

Highlighted

Failed to initialize High Availability state in ZooKeeper from Failover Controller

New Contributor

Hi Everyone,

 

A changing a Nameservice Name for Highly Available HDFS Using Cloudera Manager as described in documentation and now I can't Initializa High Avaliability State in Cloudera Manager (stack on item 6 from the documentation).

 

HDFS HA is enabled (m01, m02), ZooKeeper clients resides on m01, m02, m03 nodes, configured nameservice was removed by executing on m03: 

rmr /hadoop-ha/nameservice1

 

After "Select Actions > Initialize High Availability State in ZooKeeper" I have "Status: Failed" and next errors: 

 

Failed to initialize High Availability state in ZooKeeper for Nameservice LHTHAMHDP1: Failed to initialize High Availability state in ZooKeeper from Failover Controller (lhthdp1-m02): 16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/var/opt/teradata/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/lib/native
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:os.version=3.0.101-0.66.TDC.1.R.0-default
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:user.name=hdfs
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-hdfs
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:user.dir=/var/run/cloudera-scm-agent/process/1490-failover-controller-initialize-znode
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=lhthdp1-m01.ham.dlh.de:2181,lhthdp1-m02.ham.dlh.de:2181,lhthdp1-m03.ham.dlh.de:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@23c30a20
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: Opening socket connection to server lhthdp1-m02.ham.dlh.de/39.7.48.3:2181. Will not attempt to authenticate using SASL (unknown error)
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /39.7.48.3:57031, server: lhthdp1-m02.ham.dlh.de/39.7.48.3:2181
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: Session establishment complete on server lhthdp1-m02.ham.dlh.de/39.7.48.3:2181, sessionid = 0x253c2c5c37e081f, negotiated timeout = 5000
16/03/30 17:24:15 INFO ha.ActiveStandbyElector: Session connected.
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Session: 0x253c2c5c37e081f closed
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: EventThread shut down

 Details:

 

Create a znode in ZooKeeper inside of which the automatic failover system stores its data.
Failed to initialize High Availability state in ZooKeeper from Failover Controller (lhthdp1-m02): 16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/var/opt/teradata/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/lib/native
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:os.version=3.0.101-0.66.TDC.1.R.0-default
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:user.name=hdfs
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-hdfs
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:user.dir=/var/run/cloudera-scm-agent/process/1491-failover-controller-initialize-znode
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=lhthdp1-m01.ham.dlh.de:2181,lhthdp1-m02.ham.dlh.de:2181,lhthdp1-m03.ham.dlh.de:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@23c30a20
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: Opening socket connection to server lhthdp1-m01.ham.dlh.de/39.7.48.2:2181. Will not attempt to authenticate using SASL (unknown error)
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /39.7.48.3:54731, server: lhthdp1-m01.ham.dlh.de/39.7.48.2:2181
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: Session establishment complete on server lhthdp1-m01.ham.dlh.de/39.7.48.2:2181, sessionid = 0x153c2c5c38a081b, negotiated timeout = 5000
16/03/30 17:35:01 INFO ha.ActiveStandbyElector: Session connected.
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Session: 0x153c2c5c38a081b closed
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: EventThread shut down