Hi Ashok, to increase the file descriptors for the task tracker with a CM controlled cluster you can do the following. Keep in mind that the Alert is only a warning. Verify your current file descriptor ulimit value to your task tracker processes. pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits; Perform the following on each node in the cluster. A ulimit of 65536 probably would be enough for most use cases. I just used 262144 as an extreme example. 1. Edit /usr/sbin/cmf-agent and change the ulimit -n setting. 2. Edit /usr/lib64/cmf/service/mapreduce/mapreduce.sh and add ulimit -n 262144 at the top of the script. 3. Push these script changes out to all the slave nodes 4. A few nodes at a time in the same rack, shutdown datanode and tasktracker services on them. 5. On these stopped nodes run a: service cloudera-scm-agent hard_restart 6. Restart the datanode and tasktracker. Verify your file descriptor ulimit value to your task tracker processes after making the change. pid=$(ps faux | grep java | grep tasktracker | awk -F " " '{print $2}');cat /proc/$pid/limits; -roland
... View more
