Custom JVM properties for each flume agent

[#1]

We have flume service in HDP 2.5 which has multiple agents in it. All agents are using JVM from flume-env.sh and spawning individual JVM. Few are using less memory and few more. Is it possible to customize the JVM properties for each agent?

[#2]

@Rajesh Reddy

you can use the different flume-env.sh file for each agent and you can specify the parameters using:

export JAVA_OPTS="-Xms2048m -Xmx8196m -XX:MaxDirectMemorySize=256m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC"

Hope this helps.

[#3]

But Flume service in Ambari has only one flume-env.sh!!!

[#4]

@Rajesh Reddy

From Ambari you can only configure only one flume-env.sh file for the environment but this can achieved in two ways:

1) You can use Ambari config groups to have different flume-env.sh files for different hosts

2) If do not want to change the default config, you need to go with CLI option i.e. creating the different config files/folders for each flume agent and starting them from CLI only, but with this you will not be able to manage the flume agent's through Ambari (a limitation of Ambari)

Hope this helps.

[#5]

You could use Ambari Configuration groups to achieve that. See this thread.

[#6]

@Rajesh Reddy

Edit the flume-env.sh under Ambari to add the parameter. Restart services, this should affect all the flume agents in the cluster.