Support Questions
Find answers, ask questions, and share your expertise

What is the recommended value for the hadoop parameters ?

Highlighted

What is the recommended value for the hadoop parameters ?

Suppose if i the change value for below parameters on prod.. so what the below parameter will cause if i change the values.

HDFS

Parameter Current value PROD Current value DR

dfs.namenode.handler.count 40 100

yarn Parameter current Value(PROD) Current Value(DR)

yarn.timeline-service.ttl-ms 259200000 604800000

mapreduce

Parameter current Value(PROD) Current Value(DR)

mapreduce.task.io.sort.mb 1433 819

yarn.app.mapreduce.am.resource.mb 4096 2048

can someone suggest what value i can set and what will impact if i change the value ?

2 REPLIES 2

Re: What is the recommended value for the hadoop parameters ?

Expert Contributor

Hi @Vinay Sharma,

They are all internal values and will not probably affect your system IF they have not previously been tuned to solve a problem. I would go for the largest value in your cases and be prepared that *.mb values can affect memory consumption.

dfs.namenode.handler.count - The number of server threads for the datanode, 10 default

yarn.timeline-service.ttl-ms - Time to live for timeline store data in milliseconds. Defaults to 604800000 (7 days).

mapreduce.task.io.sort.mb - The total amount of buffer memory to use while sorting files, in megabytes. Default 100, gives each merge stream 1MB, which should minimise seeks.

yarn.app.mapreduce.am.resource.mb - The amount of memory the MR AppMaster needs, default 1536.

/Best Regards, Mats

Highlighted

Re: What is the recommended value for the hadoop parameters ?

Contributor