Created 01-21-2014 08:30 AM
Hi Ashok, to increase the file descriptors for the task tracker with a CM controlled cluster you can do the following. Keep in mind that the Alert is only a warning.
Verify your current file descriptor ulimit value to your task tracker processes.
pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits;
Perform the following on each node in the cluster. A ulimit of 65536 probably would be enough for most use cases. I just used 262144 as an extreme example.
1. Edit /usr/sbin/cmf-agent and change the ulimit -n setting.
2. Edit /usr/lib64/cmf/service/mapreduce/mapreduce.sh and add
ulimit -n 262144
at the top of the script.
3. Push these script changes out to all the slave nodes
4. A few nodes at a time in the same rack, shutdown datanode and tasktracker services on them.
5. On these stopped nodes run a: service cloudera-scm-agent hard_restart
6. Restart the datanode and tasktracker.
Verify your file descriptor ulimit value to your task tracker processes after making the change.
pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits;
-roland