Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Storm Metrics broken whenever two topologies running on one host

Storm Metrics broken whenever two topologies running on one host

Cloudera Employee

HDP's build of Storm comes bundled with a custom logback configuration file which you can find and view under https://github.com/hortonworks/storm-release/blob/HDP-2.2.6.0-tag/logback/cluster.xml. It has an appender configured called METRICS that is used whenever you configure a metrics consumer with a topology to write out topology metrics. This appender is configured to use a FixedWindowRollingPolicy, which per the Logback documentation does not support prudent mode and therefore is not capable of allowing two processes to log to the same file. This is a problem whenever two topologies run on the same host because both workers must be able to log to /var/log/storm/metrics.log. I'm not sure how to circumvent this problem. I've tried using a custom logback.xml and deploying the topology with -Dlogback.configurationFile, but that seemed to cause unintended consequences with the other log files.

2 REPLIES 2
Highlighted

Re: Storm Metrics broken whenever two topologies running on one host

Mentor
Highlighted

Re: Storm Metrics broken whenever two topologies running on one host

Cloudera Employee

@Artem Ervits

Actually this was filed to JIRA, and our team commented some approaches and now waiting for feedback.

Don't have an account?
Coming from Hortonworks? Activate your account here