Support Questions
Find answers, ask questions, and share your expertise

Ambari Flume Management Best Practices - Documentation?

Contributor

Do we have documentation available around Ambari Flume management? Based on what I know so far, this is what would make sense to me in the real world, assuming an admin does not want to be involved in setting up / maintaining / managing Flume agents:

1) Set up a separate Ambari Server instance for Flume management. (Treat Flume Configs tab like a View, in essence... if that's doable?) Thus, Flume "admins" (developers) don't have access to HDP cluster management via Ambari.

2) Add a single HDP edge node (logically speaking, at least) to the Ambari instance (Is it possible for two Ambari instances to manage different services on the same node?), as well as all other servers on which Flume agents are to be installed via Ambari add hosts.

3) Create config groups based on server identity - ex: all web servers might have identical log tracking, the HDP edge node(s) would be its (their) own config group, etc.

I don't have a test environment robust enough to try this out, so any help is appreciated. Thanks!

1 ACCEPTED SOLUTION

Contributor

The API is not an option at this point. If there's no way to do this in Ambari natively, that's fine - I just need to know that. But workarounds of any kind do not meet the need.

View solution in original post

4 REPLIES 4

Mentor

I take nifi is out of the question? @jeden

Contributor

🙂 For purposes of this question, yes.

Expert Contributor

Why dont use ambari rest api to manage Flume configs outside of main ambari screen. Personally i didnt tested it, but it should be a valid approach.

Also one issue is still there (it prevents us to use ambari for Flume managing): https://issues.apache.org/jira/browse/AMBARI-9421 And the workaround mentioned doesn't works with more than 1 agent

Contributor

The API is not an option at this point. If there's no way to do this in Ambari natively, that's fine - I just need to know that. But workarounds of any kind do not meet the need.

; ;