Issue while adding zookeeper

[#1]

Below is the error while adding zookeeper to one of the node.

+ ZOOKEEPER_SERVER_OPTS='-Djava.net.preferIPv4Stack=true -Dzookeeper.log.file=zookeeper-cmf-zookeeper-SERVER-hdp-poc-d1.vmware.com.log -Dzookeeper.log.dir=/var/log/zookeeper -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.rmi.port=9010 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djute.maxbuffer=4194304 -Dzookeeper.datadir.autocreate=false -Xms784334848 -Xmx784334848 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/zookeeper_zookeeper-SERVER-153098168435d1b55999a6d815852712_pid44416.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh'
+ exec /usr/java/jdk1.7.0_67-cloudera/bin/java -cp '/run/cloudera-scm-agent/process/375-zookeeper-server:/opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper/lib/log4j.jar:/opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper/build/*:/opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper/build/lib/*:/opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper/*:/opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper/lib/*:/usr/share/cmf/lib/plugins/event-publish-5.11.0-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-5.11.0.jar' -Djava.net.preferIPv4Stack=true -Dzookeeper.log.file=zookeeper-cmf-zookeeper-SERVER-hdp-poc-d1.vmware.com.log -Dzookeeper.log.dir=/var/log/zookeeper -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.rmi.port=9010 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djute.maxbuffer=4194304 -Dzookeeper.datadir.autocreate=false -Xms784334848 -Xmx784334848 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/zookeeper_zookeeper-SERVER-153098168435d1b55999a6d815852712_pid44416.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh org.apache.zookeeper.server.quorum.QuorumPeerMain /run/cloudera-scm-agent/process/375-zookeeper-server/zoo.cfg
Error: Exception thrown by the agent : java.lang.NullPointerException

[#2]

Is there anything useful in the cloudera-scm-agent logs on the node you are trying to add it to? Have you tried add other services to that node recently?

[#3]

@hello4usharath,

Since the stderr.log shows that the supervisor already tried starting Zookeeper and the agent/supervisor is written in python, the exception must be coming from Zookeeper itself.  I'd recommend checking the zookeeper logs on the host where it failed to start.  Also you might check "stdout.log" for the process as it sometimes contains output including role exceptions.

-Ben

[#4 · Accepted solution]

@hello4usharath,

Oh, and we have had this sort of thing happen if the port specified in "com.sun.management.jmxremote.rmi.port=9010" is already in use.  You may want to try "netstat -nap|grep 9010" to see if another process has taken that port.

Ben

[#5]

Thanks Ben,

issue got fixed.

[#6]

Hey I am getting the same error. Can you help?

[#7]

One more lastly:

"Exception thrown by the agent" is not from Cloudera's Agent; rather it is the RMI Agent I believe and a web search confirms that the error you see is when the RMI port is taken