Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎03-21-2016

Failed to initialize High Availability state in ZooKeeper from Failover Controller

Hi Everyone,

 

A changing a Nameservice Name for Highly Available HDFS Using Cloudera Manager as described in documentation and now I can't Initializa High Avaliability State in Cloudera Manager (stack on item 6 from the documentation).

 

HDFS HA is enabled (m01, m02), ZooKeeper clients resides on m01, m02, m03 nodes, configured nameservice was removed by executing on m03: 

rmr /hadoop-ha/nameservice1

 

After "Select Actions > Initialize High Availability State in ZooKeeper" I have "Status: Failed" and next errors: 

 

Failed to initialize High Availability state in ZooKeeper for Nameservice LHTHAMHDP1: Failed to initialize High Availability state in ZooKeeper from Failover Controller (lhthdp1-m02): 16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/var/opt/teradata/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/lib/native
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:os.version=3.0.101-0.66.TDC.1.R.0-default
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:user.name=hdfs
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-hdfs
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Client environment:user.dir=/var/run/cloudera-scm-agent/process/1490-failover-controller-initialize-znode
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=lhthdp1-m01.ham.dlh.de:2181,lhthdp1-m02.ham.dlh.de:2181,lhthdp1-m03.ham.dlh.de:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@23c30a20
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: Opening socket connection to server lhthdp1-m02.ham.dlh.de/39.7.48.3:2181. Will not attempt to authenticate using SASL (unknown error)
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /39.7.48.3:57031, server: lhthdp1-m02.ham.dlh.de/39.7.48.3:2181
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: Session establishment complete on server lhthdp1-m02.ham.dlh.de/39.7.48.3:2181, sessionid = 0x253c2c5c37e081f, negotiated timeout = 5000
16/03/30 17:24:15 INFO ha.ActiveStandbyElector: Session connected.
16/03/30 17:24:15 INFO zookeeper.ZooKeeper: Session: 0x253c2c5c37e081f closed
16/03/30 17:24:15 INFO zookeeper.ClientCnxn: EventThread shut down

 Details:

 

Create a znode in ZooKeeper inside of which the automatic failover system stores its data.
Failed to initialize High Availability state in ZooKeeper from Failover Controller (lhthdp1-m02): 16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/var/opt/teradata/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/lib/native
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:os.version=3.0.101-0.66.TDC.1.R.0-default
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:user.name=hdfs
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-hdfs
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Client environment:user.dir=/var/run/cloudera-scm-agent/process/1491-failover-controller-initialize-znode
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=lhthdp1-m01.ham.dlh.de:2181,lhthdp1-m02.ham.dlh.de:2181,lhthdp1-m03.ham.dlh.de:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@23c30a20
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: Opening socket connection to server lhthdp1-m01.ham.dlh.de/39.7.48.2:2181. Will not attempt to authenticate using SASL (unknown error)
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /39.7.48.3:54731, server: lhthdp1-m01.ham.dlh.de/39.7.48.2:2181
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: Session establishment complete on server lhthdp1-m01.ham.dlh.de/39.7.48.2:2181, sessionid = 0x153c2c5c38a081b, negotiated timeout = 5000
16/03/30 17:35:01 INFO ha.ActiveStandbyElector: Session connected.
16/03/30 17:35:01 INFO zookeeper.ZooKeeper: Session: 0x153c2c5c38a081b closed
16/03/30 17:35:01 INFO zookeeper.ClientCnxn: EventThread shut down