Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Custom JVM properties for each flume agent

Custom JVM properties for each flume agent

Contributor

We have flume service in HDP 2.5 which has multiple agents in it. All agents are using JVM from flume-env.sh and spawning individual JVM. Few are using less memory and few more. Is it possible to customize the JVM properties for each agent?

5 REPLIES 5

Re: Custom JVM properties for each flume agent

Expert Contributor
@Rajesh Reddy

you can use the different flume-env.sh file for each agent and you can specify the parameters using:

export JAVA_OPTS="-Xms2048m -Xmx8196m -XX:MaxDirectMemorySize=256m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC"

Hope this helps.

Re: Custom JVM properties for each flume agent

Contributor

But Flume service in Ambari has only one flume-env.sh!!!

Re: Custom JVM properties for each flume agent

Expert Contributor
@Rajesh Reddy

From Ambari you can only configure only one flume-env.sh file for the environment but this can achieved in two ways:

1) You can use Ambari config groups to have different flume-env.sh files for different hosts

2) If do not want to change the default config, you need to go with CLI option i.e. creating the different config files/folders for each flume agent and starting them from CLI only, but with this you will not be able to manage the flume agent's through Ambari (a limitation of Ambari)

Hope this helps.

Re: Custom JVM properties for each flume agent

Master Collaborator

You could use Ambari Configuration groups to achieve that. See this thread.

Re: Custom JVM properties for each flume agent

@Rajesh Reddy

Edit the flume-env.sh under Ambari to add the parameter. Restart services, this should affect all the flume agents in the cluster.

Don't have an account?
Coming from Hortonworks? Activate your account here