question Best practices for ulimits? (number of open file decriptors and processes) in Archives of Support Questions (Read Only)

Best practices for ulimits? (number of open file decriptors and processes)

amiller — Fri, 16 Sep 2022 09:45:50 GMT

What are the recommended starting values for ulimits for each component?

The Ambari 2.1 doc says 10000, but should some services be started higher, say 32k?

Is there a good way to estimate these values based on cluster size, memory/cpu, number of blocks, etc? How can we proactively adjust ulimit, to avoid waiting until a service fails because the limit is hit?

Re: Best practices for ulimits? (number of open file decriptors and processes)

bsaini — Mon, 19 Aug 2019 12:56:17 GMT

Ulimit does not depend on the size of the cluster but more on the individual node, workloads and user concurrency. I set these values as best practice, which BTW is a large enough value and is probably never reached.

* - nofile 32768

* - nproc 65536

Ambari actually changes the limit of the core files created as part of the commands executed like in this case -