TaskTracker Max open files (nofile)

[#1]

What's the best way to increase Max open files settings for tasktracker process?  The agent ignores all settings in the /etc/security/limits.d/*conf files.

[#2]

You can try setting the value in /etc/sysctl.conf
fs.epoll.max_user_instances = 2048

[#3]

There isn't a great way to fix this right now. We're looking to fix this for CM 5.

You can edit /usr/lib64/cmf/service/hdfs/hdfs.sh to change your ulimits, but know that this code does not run as root. I think processes will by default get the limits for the root user instead of the ones for the appropriate user, so you can also try changing the ulimits for root.

This issue isn't seen in many clusters, so it could also have something to do with your configuration, how you wrote the map reduce job, or maybe having a lot of small HDFS files. Could also be worth looking into, if only to optimize.

You can find a little more discussion on the topic here:

https://groups.google.com/a/cloudera.org/forum/#!topic/scm-users/G1pRUfYNOWs

Thanks,

Darren

[#4 · Accepted solution]

Thanks, but neither of these work.  I was eventually able to increase the TaskTracker limits w/

On each node

1.  sudo /etc/init.d/cloudera-scm-agent hard_stop
2.  Edit parameter in /usr/sbin/cmf-agent script.
3.  sudo /etc/init.d/cloudera-scm-agent start

Then restart each tasktracker from Cloudera console.