Thank You fol quick response, Answer of above you ask to me, root@hzk3:/usr/hdp/2.6.2.14-5/zookeeper/bin# hostname -f hzk3.mstorm.com root@hzk3:/usr/hdp/2.6.2.14-5/zookeeper/bin# netstat -tnlpa | grep 2181 tcp 0 0 10.13.10.173:51752 10.13.10.173:2181 TIME_WAIT - tcp6 0 0 :::2181 :::* LISTEN 19358/java tcp6 0 0 ::1:52474 ::1:2181 TIME_WAIT - root@hzk3:/usr/hdp/2.6.2.14-5/zookeeper/bin# echo state | nc localhost 2181 This ZooKeeper instance is not currently serving requests root@hzk3:/usr/hdp/2.6.2.14-5/zookeeper/bin# nc -v hzk3.mstorm.com 2181 Connection to hzk3.mstorm.com 2181 port [tcp/*] succeeded! ^C root@hzk3:/usr/hdp/2.6.2.14-5/zookeeper/bin# ps -ef | grep zookeeper zookeep+ 19358 1 1 14:38 ? 00:00:02 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.log.file=zookeeper-zookeeper-server-hzk3.log -Dzookeeper.root.logger=INFO,ROLLINGFILE -cp /usr/hdp/current/zookeeper-server/bin/../build/classes:/usr/hdp/current/zookeeper-server/bin/../build/lib/*.jar:/usr/hdp/current/zookeeper-server/bin/../lib/xercesMinimal-1.9.6.2.jar:/usr/hdp/current/zookeeper-server/bin/../lib/wagon-provider-api-2.4.jar:/usr/hdp/current/zookeeper-server/bin/../lib/wagon-http-shared4-2.4.jar:/usr/hdp/current/zookeeper-server/bin/../lib/wagon-http-shared-1.0-beta-6.jar:/usr/hdp/current/zookeeper-server/bin/../lib/wagon-http-lightweight-1.0-beta-6.jar:/usr/hdp/current/zookeeper-server/bin/../lib/wagon-http-2.4.jar:/usr/hdp/current/zookeeper-server/bin/../lib/wagon-file-1.0-beta-6.jar:/usr/hdp/current/zookeeper-server/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/slf4j-api-1.6.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/plexus-utils-3.0.8.jar:/usr/hdp/current/zookeeper-server/bin/../lib/plexus-interpolation-1.11.jar:/usr/hdp/current/zookeeper-server/bin/../lib/plexus-container-default-1.0-alpha-9-stable-1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/netty-3.7.0.Final.jar:/usr/hdp/current/zookeeper-server/bin/../lib/nekohtml-1.9.6.2.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-settings-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-repository-metadata-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-project-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-profile-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-plugin-registry-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-model-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-error-diagnostics-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-artifact-manager-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-artifact-2.2.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/maven-ant-tasks-2.1.3.jar:/usr/hdp/current/zookeeper-server/bin/../lib/log4j-1.2.16.jar:/usr/hdp/current/zookeeper-server/bin/../lib/jsoup-1.7.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/jline-0.9.94.jar:/usr/hdp/current/zookeeper-server/bin/../lib/commons-logging-1.1.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/commons-io-2.2.jar:/usr/hdp/current/zookeeper-server/bin/../lib/commons-codec-1.6.jar:/usr/hdp/current/zookeeper-server/bin/../lib/classworlds-1.1-alpha-2.jar:/usr/hdp/current/zookeeper-server/bin/../lib/backport-util-concurrent-3.1.jar:/usr/hdp/current/zookeeper-server/bin/../lib/ant-launcher-1.8.0.jar:/usr/hdp/current/zookeeper-server/bin/../lib/ant-1.8.0.jar:/usr/hdp/current/zookeeper-server/bin/../zookeeper-3.4.6.2.6.2.14-5.jar:/usr/hdp/current/zookeeper-server/bin/../src/java/lib/*.jar:/usr/hdp/current/zookeeper-server/conf::/usr/share/zookeeper/*:/usr/share/zookeeper/* -Xmx1024m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /usr/hdp/current/zookeeper-server/conf/zoo.cfg root 19510 3925 0 14:40 ? 00:00:00 grep --color=auto zookeeper less /var/log/zookeeper/zookeeper.log 2017-10-30 08:31:14,510 - INFO [LearnerHandler-/10.13.10.171:43240:LearnerHandler@462] - Sending SNAP 2017-10-30 08:31:14,511 - INFO [LearnerHandler-/10.13.10.171:43240:LearnerHandler@486] - Sending snapshot last zxid of peer is 0x100000380 zxid of leader is 0x100000655sent zxid of db as 0x100000655 2017-10-30 08:31:14,512 - ERROR [LearnerHandler-/10.13.10.171:43240:LearnerHandler@633] - Unexpected exception causing shutdown while sock still open java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:517) 2017-10-30 08:31:14,512 - WARN [LearnerHandler-/10.13.10.171:43240:LearnerHandler@646] - ******* GOODBYE /10.13.10.171:43240 ******** 2017-10-30 08:31:14,515 - INFO [WorkerReceiver[myid=2]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x100000380 (n.zxid), 0x2 (n.round), LOOKING (n.state), 3 (n.sid), 0x1 (n.peerEpoch) LEADING (my state) 2017-10-30 08:31:14,517 - INFO [LearnerHandler-/10.13.10.171:43242:LearnerHandler@330] - Follower sid: 3 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@7c55649 2017-10-30 08:31:14,517 - INFO [LearnerHandler-/10.13.10.171:43242:LearnerHandler@385] - Synchronizing with Follower sid: 3 maxCommittedLog=0x100000655 minCommittedLog=0x100000461 peerLastZxid=0x100000380 2017-10-30 08:31:14,517 - WARN [LearnerHandler-/10.13.10.171:43242:LearnerHandler@446] - Unhandled proposal scenario 2017-10-30 08:31:14,517 - INFO [LearnerHandler-/10.13.10.171:43242:LearnerHandler@462] - Sending SNAP 2017-10-30 08:31:14,518 - INFO [LearnerHandler-/10.13.10.171:43242:LearnerHandler@486] - Sending snapshot last zxid of peer is 0x100000380 zxid of leader is 0x100000655sent zxid of db as 0x100000655 2017-10-30 08:31:14,519 - ERROR [LearnerHandler-/10.13.10.171:43242:LearnerHandler@633] - Unexpected exception causing shutdown while sock still open java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:517) 2017-10-30 08:31:14,519 - WARN [LearnerHandler-/10.13.10.171:43242:LearnerHandler@646] - ******* GOODBYE /10.13.10.171:43242 ******** 2017-10-30 08:31:14,523 - INFO [WorkerReceiver[myid=2]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x100000380 (n.zxid), 0x2 (n.round), LOOKING (n.state), 3 (n.sid), 0x1 (n.peerEpoch) LEADING (my state) 2017-10-30 08:31:14,525 - INFO [LearnerHandler-/10.13.10.171:43244:LearnerHandler@330] - Follower sid: 3 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@7c55649 2017-10-30 08:31:14,525 - INFO [LearnerHandler-/10.13.10.171:43244:LearnerHandler@385] - Synchronizing with Follower sid: 3 maxCommittedLog=0x100000655 minCommittedLog=0x100000461 peerLastZxid=0x100000380 2017-10-30 08:31:14,526 - WARN [LearnerHandler-/10.13.10.171:43244:LearnerHandler@446] - Unhandled proposal scenario ... View more

Hello, I have checked above you mentioned. everything is correct in my case.but still I am getting same error. ... View more

Thank You for reply, In my case, I have not installed ufw and iptables. In my /var/log/ambari-metrics-grafana/grafana.out content was, Wed Oct 11 03:42:37 UTC 2017 Starting Ambari Metrics Grafana: .... Already running. Wed Oct 11 03:58:25 UTC 2017 Starting Ambari Metrics Grafana: .... Already running. I have no /var/log/ambari-metrics-grafana/grafana.out And my stdout is, 2017-10-11 03:58:25,540 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf 2017-10-11 03:58:25,626 - Stack Feature Version Info: stack_version=2.6, version=2.6.2.0-205, current_cluster_version=2.6.2.0-205 -> 2.6.2.0-205 2017-10-11 03:58:25,630 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf User Group mapping (user_group) is missing in the hostLevelParams 2017-10-11 03:58:25,631 - Group['hadoop'] {} 2017-10-11 03:58:25,631 - Group['users'] {} 2017-10-11 03:58:25,632 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']} 2017-10-11 03:58:25,633 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']} 2017-10-11 03:58:25,633 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']} 2017-10-11 03:58:25,633 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']} 2017-10-11 03:58:25,633 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']} 2017-10-11 03:58:25,634 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']} 2017-10-11 03:58:25,634 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']} 2017-10-11 03:58:25,634 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2017-10-11 03:58:25,635 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'} 2017-10-11 03:58:25,638 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if 2017-10-11 03:58:25,638 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'} 2017-10-11 03:58:25,644 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2017-10-11 03:58:25,644 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'} 2017-10-11 03:58:25,649 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if 2017-10-11 03:58:25,649 - Group['hdfs'] {} 2017-10-11 03:58:25,649 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']} 2017-10-11 03:58:25,649 - FS Type: 2017-10-11 03:58:25,649 - Directory['/etc/hadoop'] {'mode': 0755} 2017-10-11 03:58:25,657 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2017-10-11 03:58:25,657 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777} 2017-10-11 03:58:25,665 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'} 2017-10-11 03:58:25,668 - Skipping Execute[('setenforce', '0')] due to not_if 2017-10-11 03:58:25,668 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'} 2017-10-11 03:58:25,669 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'} 2017-10-11 03:58:25,669 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'} 2017-10-11 03:58:25,671 - File['/usr/hdp/current/hadoop-client/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'} 2017-10-11 03:58:25,672 - File['/usr/hdp/current/hadoop-client/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'} 2017-10-11 03:58:25,675 - File['/usr/hdp/current/hadoop-client/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644} 2017-10-11 03:58:25,680 - File['/usr/hdp/current/hadoop-client/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2017-10-11 03:58:25,681 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755} 2017-10-11 03:58:25,681 - File['/usr/hdp/current/hadoop-client/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'} 2017-10-11 03:58:25,683 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'} 2017-10-11 03:58:25,685 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755} 2017-10-11 03:58:25,804 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf 2017-10-11 03:58:25,806 - checked_call['hostid'] {} 2017-10-11 03:58:25,809 - checked_call returned (0, '0d0aa80a') 2017-10-11 03:58:25,810 - Directory['/etc/ambari-metrics-grafana/conf'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True, 'mode': 0755} 2017-10-11 03:58:25,811 - Directory['/var/log/ambari-metrics-grafana'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True, 'mode': 0755} 2017-10-11 03:58:25,812 - Directory['/var/lib/ambari-metrics-grafana'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True, 'mode': 0755} 2017-10-11 03:58:25,831 - Directory['/var/run/ambari-metrics-grafana'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True, 'mode': 0755} 2017-10-11 03:58:25,840 - File['/etc/ambari-metrics-grafana/conf/ams-grafana-env.sh'] {'content': InlineTemplate(...), 'owner': 'ams', 'group': 'hadoop'} 2017-10-11 03:58:25,843 - File['/etc/ambari-metrics-grafana/conf/ams-grafana.ini'] {'content': InlineTemplate(...), 'owner': 'ams', 'group': 'hadoop', 'mode': 0600} 2017-10-11 03:58:25,844 - Writing File['/etc/ambari-metrics-grafana/conf/ams-grafana.ini'] because contents don't match 2017-10-11 03:58:25,855 - Execute[('chown', '-R', u'ams', '/etc/ambari-metrics-grafana/conf')] {'sudo': True} 2017-10-11 03:58:25,859 - Execute[('chown', '-R', u'ams', u'/var/log/ambari-metrics-grafana')] {'sudo': True} 2017-10-11 03:58:25,863 - Execute[('chown', '-R', u'ams', u'/var/lib/ambari-metrics-grafana')] {'sudo': True} 2017-10-11 03:58:25,866 - Execute[('chown', '-R', u'ams', u'/var/run/ambari-metrics-grafana')] {'sudo': True} 2017-10-11 03:58:25,869 - Execute['/usr/sbin/ambari-metrics-grafana start'] {'not_if': "ambari-sudo.sh su ams -l -s /bin/bash -c 'test -f /var/run/ambari-metrics-grafana/grafana-server.pid && ps -p `cat /var/run/ambari-metrics-grafana/grafana-server.pid`'", 'user': 'ams'} 2017-10-11 03:58:25,890 - Grafana Server has started with pid: 31695 2017-10-11 03:58:25,890 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 03:58:45,918 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 03:58:45,918 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 03:59:05,938 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 03:59:05,938 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 03:59:25,957 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 03:59:25,957 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 03:59:45,978 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 03:59:45,978 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:00:05,998 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:00:05,998 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:00:26,018 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:00:26,018 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:00:46,039 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:00:46,039 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:01:06,054 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:01:06,055 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:01:26,067 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:01:26,067 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:01:46,075 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:01:46,075 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:02:06,096 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:02:06,096 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:02:26,114 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:02:26,114 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:02:46,135 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:02:46,136 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user 2017-10-11 04:03:06,156 - Connection to Grafana failed. Next retry in 20 seconds. 2017-10-11 04:03:06,156 - Connecting (GET) to dn5-231.mstorm.com:3000/api/user Command failed after 1 tries ... View more