Can I create multiple Flume processes on same host?

[#1]

Can I run different version of Flume with different classpath on same host?

[#2]

@Roman Sydorov

Please see this "

It is possible to run multiple Flume agents on the same host

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_installing_manually_book/content/installi...

[#3]

@Roman Sydorov

It may not be possible to manage mulitple versions with ambari.

Starting an agent

An agent is started using a shell script called flume-ng which is located in the bin directory of the Flume distribution. You need to specify the agent name, the config directory, and the config file on the command line:

$ bin/flume-ng agent -n $agent_name -c conf -f conf/flume-conf.properties.template

Now the agent will start running source and sinks configured in the given properties file.

[#4 · Accepted solution]

@Roman Sydorov I haven't tried but I don't see any reason why not. You can start each Flume version with the command below, just create separate conf directories and set respective classpaths in flume-env.sh. Each one can run multiple agents and each agent will run as a separate process. However, from Ambari you can start and manage only one Flume version specified by default in /etc/flume/conf.

/usr/hdp/current/flume-server/bin/flume-ng agent -c /etc/flume/conf -f /etc/flume/conf/flume.conf -n agent

[#5]

Ambari run each agent in new process. It split single configuration (on ambari screen) to multiple folders (per agent) on the node

[#6]

@Michael M thanks for clarification. @Roman Sydorov Please see my edited answer above.