Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

limits.conf limits.conf.d/* best practices

avatar
New Contributor

Hi All,

 

We are looking for some best practice guidelines on setting limits for users in the limits.conf.

We don't use the "*" for username in the limits.conf, they must be defined. CDP is the only thing on our clusters and would like to know what you suggest for each of the service users (i.e. hive/spark/hdfs/oozie, yarn, httpfs, clouder-scm, sqoop, knox, livy, flume, hue, zookeeper, mapred, druid, ranger, solr, etc,  essentially a breakdown of all users that can be added as part of the CDP environment.

I have looked through the documentation and haven't found a definitive answer to this question.

 

Any help deeply appreciated,

 

Thanks in advance,

1 REPLY 1

avatar
Master Guru

@innominate Please refer the below doc that will give you some idea. 

Cloudera Manager automatically sets nproc configuration in /etc/security/limits.conf, but this configuration can be overridden by individual files in /etc/security/limits.d/. This can cause problems with Apache Impala and other components.

Make sure that the nproc limits are set sufficiently high, such as 65536 or 262144.

1. https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_os_requirements.html 2. https://docs.cloudera.com/runtime/7.2.9/configuring-hbase/topics/hbase-configure-ulimit-pam-cli.html


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.