Created on 05-02-2016 09:03 AM - edited 08-19-2019 03:10 AM
Hello everyone.
I have succesfully started flume agent via shell, but I faced with troubles while started agent in Ambari.
I did as explained here: http://cleverowl.uk/2015/09/30/ingesting-files-with-apache-flume/
I added an agent to Ambari, but it didn't doing anything.
Created 05-02-2016 01:18 PM
The issue was commonplace:unsufficient permissions on directories.
Thank you everyone!
Created 05-02-2016 09:07 AM
Created 05-02-2016 09:30 AM
1. Yes, I set up a Flume agent using Ambari like this article
2. Where can I found logs for Ambari Flume agents? /var/log/flume/flume.log doesn't contain any info about this agent.
Created 05-02-2016 11:03 AM
I assume you are using the ambari-metrics-system to collect statistics. You need to add a jar to the flume classpath in order to make the charts work. Edit the 'Advanced flume-env' config in Ambari.
Make sure that the flume-env template contains:
... if [ -e "/usr/lib/flume/lib/ambari-metrics-flume-sink.jar" ]; then export FLUME_CLASSPATH=$FLUME_CLASSPATH:/usr/lib/flume/lib/ambari-metrics-flume-sink.jar fi ...
Restart flume, now you should be able to see the collected metrics.
Created 05-02-2016 11:24 AM
Thank you for responce.
My 'Advanced flume-env' config in Ambari already contain this section.
The problem is not only in the lack of metrics, but also in the fact that the agent does not run through Ambari, and does not handle files. Although through the shell everything works well
flume-ng agent --conf conf --conf-file /home/ambari-qa/flume/example4.conf --name agent -Dflume.root.logger=INFO,console
Created 05-02-2016 01:18 PM
The issue was commonplace:unsufficient permissions on directories.
Thank you everyone!