Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

No Workers in Storm for Squid Topology

avatar
New Contributor

I'm attempting to follow the new data source tutorial for Metron. I have squid up and running, the topology was added to Storm, the topic added to kafka, NiFi is setup and everything seems to be correct, but it is not working.

When I look at the Storm UI, there are no workers running for the Squid topology. I'd assume I'd see one, since the config states 1. There are entries in the /data1/kafka-log/squid-0/ directory for the log files, but that's the only place I see any related data from the squidclient entries.

Any idea where I can start looking?

Possibly related question, after I installed Metron I got a handful of records in the UI. Since then, nothing is being shown. The system has been up for an hour and it's only displaying ~15 minutes of data?

1 ACCEPTED SOLUTION

avatar
New Contributor

I figured this out. It was due to the configuration of the Storm cluster. I didn't realize the cluster had a max of 4 workers set and each worker had already been committed to a topology.

View solution in original post

5 REPLIES 5

avatar
New Contributor

I figured this out. It was due to the configuration of the Storm cluster. I didn't realize the cluster had a max of 4 workers set and each worker had already been committed to a topology.

avatar
Contributor

@Rob Becker How did you resolve this issue? Someone suggested either deactivating or killing another of the topologies to free up a worker. I tried both and neither works. Did you add another port? If so, how? Thanks for your help.

avatar
New Contributor

@Rita McKissick I ended up removing one of the other topologies. I think I ran "storm kill <topology>". My implementation of Metron is only going to take advantage of the enrichment topology. I removed bro and the other ones that I'm not going to use.

avatar
Rising Star

You can also add an additional storm worker in Ambari -> Storm -> Configs -> supervisor.slot.ports by assigning an additional port to the list

avatar
Explorer

Yes KC. Adding an extra port for squid has worked for me . Thanks.