Support Questions

Find answers, ask questions, and share your expertise

Metron Storm Worker Hostname Issues

avatar
Rising Star

I have Metron deployed on a single node on AWS. Recently had to update the hostname to use AWS private DNS instead of the public DNS (which changes with each reboot). I think I have got most of the services working after the update but I have still have some issues with Storm. A sample of the storm worker logs are copied below. In particular, the o.a.h.i.Client seems to be still referring to the old public EC2 domain name but I have been unable to figure out where that config is specified. Could someone assist in pointing me to where that particular variable is stored?

2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] Deleted partition managers: []
2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] New partition managers: []
2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] Finished refreshing
2016-07-27 06:41:22.253 b.s.m.n.Server [INFO] Getting metrics for server on port 6704
2016-07-27 06:41:24.037 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 32 time(s); maxRetries=45
2016-07-27 06:41:44.058 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 33 time(s); maxRetries=45
2016-07-27 06:42:04.078 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 34 time(s); maxRetries=45
2016-07-27 06:42:21.626 s.k.ZkCoordinator [INFO] Task [1/1] Refreshing partition manager connections
2016-07-27 06:42:21.627 s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{partitionMap={0=ip-10-0-0-21.us-west-2.compute.internal:6667}}
2016-07-27 06:42:21.627 s.k.KafkaUtils [INFO] Task [1/1] assigned [Partition{host=ip-10-0-0-21.us-west-2.compute.internal:6667, partition=0}]
2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] Deleted partition managers: []
2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] New partition managers: []
2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] Finished refreshing
2016-07-27 06:42:22.254 b.s.m.n.Server [INFO] Getting metrics for server on port 6704
2016-07-27 06:42:24.104 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 35 time(s); maxRetries=45
2016-07-27 06:42:44.121 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 36 time(s); maxRetries=45
2016-07-27 06:43:04.139 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 37 time(s); maxRetries=45
2016-07-27 06:43:21.629 s.k.ZkCoordinator [INFO] Task [1/1] Refreshing partition manager connections
2016-07-27 06:43:21.630 s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{partitionMap={0=ip-10-0-0-21.us-west-2.compute.internal:6667}}
2016-07-27 06:43:21.631 s.k.KafkaUtils [INFO] Task [1/1] assigned [Partition{host=ip-10-0-0-21.us-west-2.compute.internal:6667, partition=0}]
1 ACCEPTED SOLUTION

avatar
Rising Star

Ok turns out that core-site.xml, hbase-site.xml and hdfs-site.xml are embedded in the metron jars at /usr/metron/0.2.0BETA/lib so needed to update the hostname references in those files to get it working again.

View solution in original post

1 REPLY 1

avatar
Rising Star

Ok turns out that core-site.xml, hbase-site.xml and hdfs-site.xml are embedded in the metron jars at /usr/metron/0.2.0BETA/lib so needed to update the hostname references in those files to get it working again.