Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

No Workers in Storm for Squid Topology

Solved Go to solution
Highlighted

No Workers in Storm for Squid Topology

New Contributor

I'm attempting to follow the new data source tutorial for Metron. I have squid up and running, the topology was added to Storm, the topic added to kafka, NiFi is setup and everything seems to be correct, but it is not working.

When I look at the Storm UI, there are no workers running for the Squid topology. I'd assume I'd see one, since the config states 1. There are entries in the /data1/kafka-log/squid-0/ directory for the log files, but that's the only place I see any related data from the squidclient entries.

Any idea where I can start looking?

Possibly related question, after I installed Metron I got a handful of records in the UI. Since then, nothing is being shown. The system has been up for an hour and it's only displaying ~15 minutes of data?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: No Workers in Storm for Squid Topology

New Contributor

I figured this out. It was due to the configuration of the Storm cluster. I didn't realize the cluster had a max of 4 workers set and each worker had already been committed to a topology.

5 REPLIES 5

Re: No Workers in Storm for Squid Topology

New Contributor

I figured this out. It was due to the configuration of the Storm cluster. I didn't realize the cluster had a max of 4 workers set and each worker had already been committed to a topology.

Re: No Workers in Storm for Squid Topology

New Contributor

@Rob Becker How did you resolve this issue? Someone suggested either deactivating or killing another of the topologies to free up a worker. I tried both and neither works. Did you add another port? If so, how? Thanks for your help.

Re: No Workers in Storm for Squid Topology

New Contributor

@Rita McKissick I ended up removing one of the other topologies. I think I ran "storm kill <topology>". My implementation of Metron is only going to take advantage of the enrichment topology. I removed bro and the other ones that I'm not going to use.

Re: No Workers in Storm for Squid Topology

Contributor

You can also add an additional storm worker in Ambari -> Storm -> Configs -> supervisor.slot.ports by assigning an additional port to the list

Re: No Workers in Storm for Squid Topology

New Contributor

Yes KC. Adding an extra port for squid has worked for me . Thanks.