question Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect in Archives of Support Questions (Read Only)

Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

rainer_geissend — Sun, 07 Feb 2016 02:54:01 GMT

Dear All, just trying to run the tutorial and everything works fine until I start the Kafka Producer.

My Setup: HDP-2.3.4.0-3485 on a 3N Yarn Setup

I am getting the following error in the k-event-processor-3-1454777987-worker-6700.log

2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.version=3.10.0-229.14.1.el7.x86_64
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.name=storm
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/storm
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.dir=/home/storm
2016-02-06 17:28:05.221 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=.....:2181,....:2181,....:2181, sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@209d0f1c
2016-02-06 17:28:05.283 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-02-06 17:28:05.286 o.a.z.ClientCnxn [INFO] Opening socket connection to server .....:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.286 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@33745cd
2016-02-06 17:28:05.287 o.a.z.ClientCnxn [INFO] Socket connection established to .....:2181, initiating session
2016-02-06 17:28:05.318 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.319 o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_60]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_60]
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) ~[zookeeper-3.4.6.2.3.4.0-3485.jar:3.4.6-3485--1]
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) [zookeeper-3.4.6.2.3.4.0-3485.jar:3.4.6-3485--1]
2016-02-06 17:28:05.340 o.a.z.ClientCnxn [INFO] Session establishment complete on server hdp1n1.aye1vpcdev/10.0.202.157:2181, sessionid = 0x152b6fb9a45005e, negotiated timeout = 20000
2016-02-06 17:28:05.345 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-02-06 17:28:05.434 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.435 o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

Any Idea what I am doing wrong?

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

nsabharwal — Sun, 07 Feb 2016 02:54:02 GMT

@Rainer Geissendoerfer

See this

Opening socket connection to server localhost/127.0.0.1:2181.Willnot attempt to authenticate using SASL (unknown error)

2016-02-0617:28:05.319 o.a.z.ClientCnxn[WARN]Session0x0for server null, unexpected error, closing socket connection and attempting reconnect

java.net.ConnectException:Connection refused

Looks like that , zookeeper is not running in localhost.

Go to Ambari and find zookeeper server address.

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

rainer_geissend — Sun, 07 Feb 2016 04:31:49 GMT

Thanks Neeraj, though I don't understand the following: my storm nimbus is installed and running on a node where zookeeper is running on default port 2181. As I am using the storm topology from the tutorial (and I assume the topology is running from the nimbus node) does that mean the zookeeper address is somewhat hardcoded in the topology? ... I deployed the topology though from a node where zookeeper is not running (if that matters) ... As I am just starting with Storm playing around I might just have a lack of understanding for how things working together, so sorry for that!

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

nsabharwal — Sun, 07 Feb 2016 04:48:25 GMT

@Rainer Geissendoerfer The easy solution is "install zk server in local node/storm topolgy running"

What's the tutorrial link?

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

aervits — Mon, 19 Aug 2019 09:45:37 GMT

@Rainer Geissendoerfer

you need to tell storm the zookeeper quorum. So go to storm config, search for the property below and specify your zookeeper host if only one or quorum if 1+ (comma separated list of zookeeper servers). Look at the screenshot.

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

nsabharwal — Sun, 07 Feb 2016 04:54:48 GMT

@Rainer Geissendoerfer You do have to tweak setting when run tutorial based on sandbox in multinode cluster 😉

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

rainer_geissend — Sun, 07 Feb 2016 05:52:41 GMT

storm.zookeeper.servers in the storm config is set to my three servers running zookeeper (as per Ambaris default) so unfortunately this isn't the solution 😞

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

rainer_geissend — Sun, 07 Feb 2016 05:55:02 GMT

http://hortonworks.com/hadoop-tutorial/simulating-transporting-realtime-events-stream-apache-kafka/#section_4

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

rainer_geissend — Sun, 07 Feb 2016 05:55:32 GMT

I am getting into it ... 🙂

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

nsabharwal — Sun, 07 Feb 2016 05:58:15 GMT

thanks @Rainer Geissendoerfer for sharing the link..Details are in the tutorial.

Looking forward to see this working 🙂

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

nsabharwal — Sun, 07 Feb 2016 05:59:14 GMT

@Rainer Geissendoerfer As it's in multinode cluster/non sandbox so I wont set sandbox entries in /etc/host as it can be very confusing.

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

rainer_geissend — Sun, 07 Feb 2016 15:57:38 GMT

Dear All, I found the solution to the problem ... zookeeper has to be installed on the nodes the supervisors are running on ... why that is? I don't know ... thanks for your support!

Re: Processing Real-time events with Apache Storm - Tutorial ... Spout cannot connect

nsabharwal — Sun, 07 Feb 2016 19:46:57 GMT

@Rainer Geissendoerfer Thats the easy fix and its because somewhere in the code , it may be looking for localhost:2181