question Re: No Workers in Storm for Squid Topology in Archives of Support Questions (Read Only)

No Workers in Storm for Squid Topology

rob_becker — Tue, 10 May 2016 21:04:57 GMT

I'm attempting to follow the new data source tutorial for Metron. I have squid up and running, the topology was added to Storm, the topic added to kafka, NiFi is setup and everything seems to be correct, but it is not working.

When I look at the Storm UI, there are no workers running for the Squid topology. I'd assume I'd see one, since the config states 1. There are entries in the /data1/kafka-log/squid-0/ directory for the log files, but that's the only place I see any related data from the squidclient entries.

Any idea where I can start looking?

Possibly related question, after I installed Metron I got a handful of records in the UI. Since then, nothing is being shown. The system has been up for an hour and it's only displaying ~15 minutes of data?

Re: No Workers in Storm for Squid Topology

rob_becker — Wed, 11 May 2016 00:28:58 GMT

I figured this out. It was due to the configuration of the Storm cluster. I didn't realize the cluster had a max of 4 workers set and each worker had already been committed to a topology.

Re: No Workers in Storm for Squid Topology

rmckissick — Thu, 26 May 2016 00:15:58 GMT

@Rob Becker How did you resolve this issue? Someone suggested either deactivating or killing another of the topologies to free up a worker. I tried both and neither works. Did you add another port? If so, how? Thanks for your help.

Re: No Workers in Storm for Squid Topology

rob_becker — Thu, 26 May 2016 02:38:36 GMT

@Rita McKissick I ended up removing one of the other topologies. I think I ran "storm kill <topology>". My implementation of Metron is only going to take advantage of the enrichment topology. I removed bro and the other ones that I'm not going to use.

Re: No Workers in Storm for Squid Topology

kctan6 — Wed, 13 Jul 2016 11:11:31 GMT

You can also add an additional storm worker in Ambari -> Storm -> Configs -> supervisor.slot.ports by assigning an additional port to the list

Re: No Workers in Storm for Squid Topology

sshahane — Thu, 11 Aug 2016 19:22:43 GMT

Yes KC. Adding an extra port for squid has worked for me . Thanks.