Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Who agreed with this solution

avatar
Expert Contributor

Hi Ashok, to increase the file descriptors for the task tracker with a CM controlled cluster you can do the following. Keep in mind that the Alert is only a warning.

 

Verify your current file descriptor ulimit value to your task tracker processes.

 

pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits;


Perform the following on each node in the cluster. A ulimit of 65536 probably would be enough for most use cases. I just used 262144 as an extreme example.

1. Edit /usr/sbin/cmf-agent and change the ulimit -n setting. 
2. Edit /usr/lib64/cmf/service/mapreduce/mapreduce.sh and add 

ulimit -n 262144 

at the top of the script. 
3. Push these script changes out to all the slave nodes 
4. A few nodes at a time in the same rack, shutdown datanode and tasktracker services on them. 
5. On these stopped nodes run a: service cloudera-scm-agent hard_restart 
6. Restart the datanode and tasktracker.

 

Verify your file descriptor ulimit value to your task tracker processes after making the change.

 

pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits;

 

-roland

 

View solution in original post

Who agreed with this solution