Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to start flume agent with the monitor port on?

Solved Go to solution

how to start flume agent with the monitor port on?

New Contributor

The open source flume can start with a monitor port on, for searching flume state. In ambari, the start command is:

{flume_bin} agent --name {{0}} --conf {{1}} --conf-file {{2}}{{3}} > {flume_log_dir}/{{4}}.out 2>&1

Is it possible to add a properties:

-Dflume.monitoring.type=http -D flume.monitoring.port=xxx
1 ACCEPTED SOLUTION

Accepted Solutions

Re: how to start flume agent with the monitor port on?

@Lei Tang,

Currently Ambari doesn't allow passing monitoring type and port from GUI. You have to pass it through command line.

Below is the snippet from (/var/lib/ambari-server/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py). You can see that monitoring type is added only when AMS is enabled.

if params.has_metric_collector:
  extra_args = '-Dflume.monitoring.type=org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink ' \
               '-Dflume.monitoring.node={0}:{1}'
  # TODO check if this is used.
  extra_args = extra_args.format(params.metric_collector_host, params.metric_collector_port)

If you want to hack it, you can put the values here in the file and restart ambari server .

Thanks,

Aditya

5 REPLIES 5

Re: how to start flume agent with the monitor port on?

@Lei Tang,

Yes. You can add these properties. Check this link for more info

Thanks,

Aditya

Re: how to start flume agent with the monitor port on?

New Contributor

Thanks.

I wander if there are any configurations to run flume agent using ambari to enable monitoring. Or this is not application scenario of ambari.

Re: how to start flume agent with the monitor port on?

@Lei Tang,

Currently Ambari doesn't allow passing monitoring type and port from GUI. You have to pass it through command line.

Below is the snippet from (/var/lib/ambari-server/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py). You can see that monitoring type is added only when AMS is enabled.

if params.has_metric_collector:
  extra_args = '-Dflume.monitoring.type=org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink ' \
               '-Dflume.monitoring.node={0}:{1}'
  # TODO check if this is used.
  extra_args = extra_args.format(params.metric_collector_host, params.metric_collector_port)

If you want to hack it, you can put the values here in the file and restart ambari server .

Thanks,

Aditya

Re: how to start flume agent with the monitor port on?

New Contributor

Thank you for your help.

Re: how to start flume agent with the monitor port on?

@Lei Tang,

Glad that it helped. Can you please accept the answer by clicking "Accept" button. This will be really useful for other community users to find the answer quickly