question Re: Ambari Flume Management Best Practices - Documentation? in Archives of Support Questions (Read Only)

Ambari Flume Management Best Practices - Documentation?

jeden — Thu, 04 Feb 2016 02:00:30 GMT

Do we have documentation available around Ambari Flume management? Based on what I know so far, this is what would make sense to me in the real world, assuming an admin does not want to be involved in setting up / maintaining / managing Flume agents:

1) Set up a separate Ambari Server instance for Flume management. (Treat Flume Configs tab like a View, in essence... if that's doable?) Thus, Flume "admins" (developers) don't have access to HDP cluster management via Ambari.

2) Add a single HDP edge node (logically speaking, at least) to the Ambari instance (Is it possible for two Ambari instances to manage different services on the same node?), as well as all other servers on which Flume agents are to be installed via Ambari add hosts.

3) Create config groups based on server identity - ex: all web servers might have identical log tracking, the HDP edge node(s) would be its (their) own config group, etc.

I don't have a test environment robust enough to try this out, so any help is appreciated. Thanks!

Re: Ambari Flume Management Best Practices - Documentation?

aervits — Thu, 04 Feb 2016 02:02:03 GMT

I take nifi is out of the question? @jeden

Re: Ambari Flume Management Best Practices - Documentation?

jeden — Thu, 04 Feb 2016 02:03:40 GMT

🙂 For purposes of this question, yes.

Re: Ambari Flume Management Best Practices - Documentation?

bluesmix — Thu, 04 Feb 2016 02:44:11 GMT

Why dont use ambari rest api to manage Flume configs outside of main ambari screen. Personally i didnt tested it, but it should be a valid approach.

Also one issue is still there (it prevents us to use ambari for Flume managing): https://issues.apache.org/jira/browse/AMBARI-9421 And the workaround mentioned doesn't works with more than 1 agent

Re: Ambari Flume Management Best Practices - Documentation?

jeden — Thu, 04 Feb 2016 05:19:00 GMT

The API is not an option at this point. If there's no way to do this in Ambari natively, that's fine - I just need to know that. But workarounds of any kind do not meet the need.