Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

[#1]

Dear All, just trying to run the tutorial and everything works fine until I start the Kafka Producer.

My Setup: HDP-2.3.4.0-3485 on a 3N Yarn Setup

I am getting the following error in the k-event-processor-3-1454777987-worker-6700.log

2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.version=3.10.0-229.14.1.el7.x86_64
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.name=storm
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/storm
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.dir=/home/storm
2016-02-06 17:28:05.221 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=.....:2181,....:2181,....:2181, sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@209d0f1c
2016-02-06 17:28:05.283 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-02-06 17:28:05.286 o.a.z.ClientCnxn [INFO] Opening socket connection to server .....:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.286 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@33745cd
2016-02-06 17:28:05.287 o.a.z.ClientCnxn [INFO] Socket connection established to .....:2181, initiating session
2016-02-06 17:28:05.318 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.319 o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_60]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_60]
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) ~[zookeeper-3.4.6.2.3.4.0-3485.jar:3.4.6-3485--1]
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) [zookeeper-3.4.6.2.3.4.0-3485.jar:3.4.6-3485--1]
2016-02-06 17:28:05.340 o.a.z.ClientCnxn [INFO] Session establishment complete on server hdp1n1.aye1vpcdev/10.0.202.157:2181, sessionid = 0x152b6fb9a45005e, negotiated timeout = 20000
2016-02-06 17:28:05.345 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-02-06 17:28:05.434 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.435 o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

Any Idea what I am doing wrong?

[#2]

@Rainer Geissendoerfer

See this

Opening socket connection to server localhost/127.0.0.1:2181.Willnot attempt to authenticate using SASL (unknown error)

2016-02-0617:28:05.319 o.a.z.ClientCnxn[WARN]Session0x0for server null, unexpected error, closing socket connection and attempting reconnect

java.net.ConnectException:Connection refused

Looks like that , zookeeper is not running in localhost.

Go to Ambari and find zookeeper server address.

[#3]

Thanks Neeraj, though I don't understand the following: my storm nimbus is installed and running on a node where zookeeper is running on default port 2181. As I am using the storm topology from the tutorial (and I assume the topology is running from the nimbus node) does that mean the zookeeper address is somewhat hardcoded in the topology? ... I deployed the topology though from a node where zookeeper is not running (if that matters) ... As I am just starting with Storm playing around I might just have a lack of understanding for how things working together, so sorry for that!

[#4]

@Rainer Geissendoerfer The easy solution is "install zk server in local node/storm topolgy running"

What's the tutorrial link?

[#5]

@Rainer Geissendoerfer You do have to tweak setting when run tutorial based on sandbox in multinode cluster 😉

[#6]

I am getting into it ... 🙂

[#7]

thanks @Rainer Geissendoerfer for sharing the link..Details are in the tutorial.

Looking forward to see this working 🙂

[#8]

http://hortonworks.com/hadoop-tutorial/simulating-transporting-realtime-events-stream-apache-kafka/#...

[#9]

@Rainer Geissendoerfer As it's in multinode cluster/non sandbox so I wont set sandbox entries in /etc/host as it can be very confusing.

[#10]

@Rainer Geissendoerfer

you need to tell storm the zookeeper quorum. So go to storm config, search for the property below and specify your zookeeper host if only one or quorum if 1+ (comma separated list of zookeeper servers). Look at the screenshot.