We are looking for some best practice guidelines on setting limits for users in the limits.conf.
We don't use the "*" for username in the limits.conf, they must be defined. CDP is the only thing on our clusters and would like to know what you suggest for each of the service users (i.e. hive/spark/hdfs/oozie, yarn, httpfs, clouder-scm, sqoop, knox, livy, flume, hue, zookeeper, mapred, druid, ranger, solr, etc, essentially a breakdown of all users that can be added as part of the CDP environment.
I have looked through the documentation and haven't found a definitive answer to this question.
@innominate Please refer the below doc that will give you some idea.
Cloudera Manager automatically setsnprocconfiguration in/etc/security/limits.conf, but this configuration can be overridden by individual files in/etc/security/limits.d/. This can causeproblemswith Apache Impala and other components.
Make sure that thenproclimits are set sufficiently high, such as65536or262144.