2016-10-14 08:09:01,588 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.library.path=:/usr/hdp/2.4.2.0-258/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.4.2.0-258/hadoop/lib/native:/usr/hdp/2.4.2.0-258/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.4.2.0-258/hadoop/lib/native

2016-10-14 08:09:01,588 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.io.tmpdir=/tmp

2016-10-14 08:09:01,588 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.compiler=<NA>

2016-10-14 08:09:01,588 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.name=Linux

2016-10-14 08:09:01,588 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.arch=amd64

2016-10-14 08:09:01,589 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.version=2.6.32-642.1.1.el6.x86_64

2016-10-14 08:09:01,589 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.name=hdfs

2016-10-14 08:09:01,591 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.home=/home/hdfs

2016-10-14 08:09:01,591 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.dir=/usr/hdp/2.4.2.0-258/hadoop

2016-10-14 08:09:01,591 INFO zookeeper.ZooKeeper (ZooKeeper.java:<init>(438)) - Initiating client connection, connectString=<standby namenode hostname>,<zookeeper server hostname>:2181,<namenode hostname>:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@a8eb1a8

2016-10-14 08:09:01,788 INFO zookeeper.Login (Login.java:login(293)) - successfully logged in.

2016-10-14 08:09:01,793 INFO client.ZooKeeperSaslClient (ZooKeeperSaslClient.java:run(285)) - Client will use GSSAPI as SASL mechanism.

2016-10-14 08:09:01,795 INFO zookeeper.Login (Login.java:run(127)) - TGT refresh thread started.

2016-10-14 08:09:01,805 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server <zookeeper server hostname>/10.150.32.95:2181. Will attempt to SASL-authenticate using Login Context section 'Client'

2016-10-14 08:09:01,809 INFO zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(864)) - Socket connection established to <zookeeper server hostname>/10.150.32.95:2181, initiating session

2016-10-14 08:09:01,811 INFO zookeeper.Login (Login.java:getRefreshTime(301)) - TGT valid starting at: Fri Oct 14 08:09:01 EDT 2016

2016-10-14 08:09:01,811 INFO zookeeper.Login (Login.java:getRefreshTime(302)) - TGT expires: Sat Oct 15 08:09:01 EDT 2016

2016-10-14 08:09:01,812 INFO zookeeper.Login (Login.java:run(181)) - TGT refresh sleeping until: Sat Oct 15 04:29:23 EDT 2016

2016-10-14 08:09:01,837 INFO zookeeper.ClientCnxn (ClientCnxn.java:onConnected(1279)) - Session establishment complete on server <zookeeper server hostname>/10.150.32.95:2181, sessionid = 0x357c317bf070000, negotiated timeout = 5000

2016-10-14 08:09:01,839 INFO ha.ActiveStandbyElector (ActiveStandbyElector.java:processWatchEvent(595)) - Session connected.

2016-10-14 08:09:01,859 ERROR client.ZooKeeperSaslClient (ZooKeeperSaslClient.java:respondToServer(323)) - SASL authentication failed using login context 'Client'.

2016-10-14 08:09:01,967 FATAL ha.ActiveStandbyElector (ActiveStandbyElector.java:fatalError(690)) - Unexpected Zookeeper watch event state: AuthFailed

2016-10-14 08:09:01,968 INFO ha.ActiveStandbyElector (ActiveStandbyElector.java:terminateConnection(830)) - Terminating ZK connection for elector id=724131351 appData=null cb=Elector callbacks for NameNode at <namenode hostname>/10.150.32.94:8020

2016-10-14 08:09:01,968 FATAL ha.ZKFailoverController (ZKFailoverController.java:fatalError(374)) - Fatal error occurred:Unexpected Zookeeper watch event state: AuthFailed

2016-10-14 08:09:01,968 INFO ha.ActiveStandbyElector (ActiveStandbyElector.java:terminateConnection(827)) - terminateConnection, zkConnectionState = TERMINATED

2016-10-14 08:09:01,969 FATAL tools.DFSZKFailoverController (DFSZKFailoverController.java:main(185)) - Got a fatal error, exiting now

java.io.IOException: Couldn't determine existence of znode '/hadoop-ha/dshdp-rlab'

at org.apache.hadoop.ha.ActiveStandbyElector.parentZNodeExists(ActiveStandbyElector.java:319)

at org.apache.hadoop.ha.ZKFailoverController.doRun(ZKFailoverController.java:218)

at org.apache.hadoop.ha.ZKFailoverController.access$000(ZKFailoverController.java:61)

at org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:172)

at org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:168)

at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:449)

at org.apache.hadoop.ha.ZKFailoverController.run(ZKFailoverController.java:168)

at org.apache.hadoop.hdfs.tools.DFSZKFailoverController.main(DFSZKFailoverController.java:183)

Caused by: org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /hadoop-ha/dshdp-rlab

at org.apache.zookeeper.KeeperException.create(KeeperException.java:123)

at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)

at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)

at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1073)

at org.apache.hadoop.ha.ActiveStandbyElector.parentZNodeExists(ActiveStandbyElector.java:317)