Support Questions

Find answers, ask questions, and share your expertise

Who agreed with this solution

avatar
Expert Contributor

Hi Ashok, to increase the file descriptors for the task tracker with a CM controlled cluster you can do the following. Keep in mind that the Alert is only a warning.

 

Verify your current file descriptor ulimit value to your task tracker processes.

 

pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits;


Perform the following on each node in the cluster. A ulimit of 65536 probably would be enough for most use cases. I just used 262144 as an extreme example.

1. Edit /usr/sbin/cmf-agent and change the ulimit -n setting. 
2. Edit /usr/lib64/cmf/service/mapreduce/mapreduce.sh and add 

ulimit -n 262144 

at the top of the script. 
3. Push these script changes out to all the slave nodes 
4. A few nodes at a time in the same rack, shutdown datanode and tasktracker services on them. 
5. On these stopped nodes run a: service cloudera-scm-agent hard_restart 
6. Restart the datanode and tasktracker.

 

Verify your file descriptor ulimit value to your task tracker processes after making the change.

 

pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits;

 

-roland

 

View solution in original post

Who agreed with this solution