Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to start flume agent with the monitor port on?

avatar
New Contributor

The open source flume can start with a monitor port on, for searching flume state. In ambari, the start command is:

{flume_bin} agent --name {{0}} --conf {{1}} --conf-file {{2}}{{3}} > {flume_log_dir}/{{4}}.out 2>&1

Is it possible to add a properties:

-Dflume.monitoring.type=http -D flume.monitoring.port=xxx
1 ACCEPTED SOLUTION

avatar
Super Guru

@Lei Tang,

Currently Ambari doesn't allow passing monitoring type and port from GUI. You have to pass it through command line.

Below is the snippet from (/var/lib/ambari-server/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py). You can see that monitoring type is added only when AMS is enabled.

if params.has_metric_collector:
  extra_args = '-Dflume.monitoring.type=org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink ' \
               '-Dflume.monitoring.node={0}:{1}'
  # TODO check if this is used.
  extra_args = extra_args.format(params.metric_collector_host, params.metric_collector_port)

If you want to hack it, you can put the values here in the file and restart ambari server .

Thanks,

Aditya

View solution in original post

5 REPLIES 5

avatar
Super Guru

@Lei Tang,

Yes. You can add these properties. Check this link for more info

Thanks,

Aditya

avatar
New Contributor

Thanks.

I wander if there are any configurations to run flume agent using ambari to enable monitoring. Or this is not application scenario of ambari.

avatar
Super Guru

@Lei Tang,

Currently Ambari doesn't allow passing monitoring type and port from GUI. You have to pass it through command line.

Below is the snippet from (/var/lib/ambari-server/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py). You can see that monitoring type is added only when AMS is enabled.

if params.has_metric_collector:
  extra_args = '-Dflume.monitoring.type=org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink ' \
               '-Dflume.monitoring.node={0}:{1}'
  # TODO check if this is used.
  extra_args = extra_args.format(params.metric_collector_host, params.metric_collector_port)

If you want to hack it, you can put the values here in the file and restart ambari server .

Thanks,

Aditya

avatar
New Contributor

Thank you for your help.

avatar
Super Guru

@Lei Tang,

Glad that it helped. Can you please accept the answer by clicking "Accept" button. This will be really useful for other community users to find the answer quickly