Adding on top of Ben's suggestion. In this link you can find additional info on requirements for container-executor mount https://docs.cloudera.com/runtime/7.0.3/yarn-security/topics/yarn-linux-container-executor.html Quote from Doc: make sure the mount point for the parcel directory is without the nosuid option. The container-executor program must have a very specific set of permissions and ownership to function correctly. In particular, it must: Be owned by root. Be owned by a group that contains only the user running the YARN daemons. Be setuid. Be group readable and executable. This corresponds to the ownership root:yarn and the permissions 6050     ... View more

Can you share the result of  ls -nl /var/lib/yarn-ce/bin/container-executor This is to make sure the uid of yarn matches the uid on the container-executor ... View more

This property is hive specific so it can be either enabled on hiveserver2 server level or in the client session level but is not possible to set on queue level (coz scheduler dont understand hive specific properties). If you set this property in hiveserver2 level - then all hive client will be mandated to use where clause when user run query on partitioned table.   Other property which you can use is hive.metastore.limit.partition.request  https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/admin_hive_tuning.html Quoting from link Set the hive.metastore.limit.partition.request parameter to 1000 to limit the maximum number of partitions accessed from a single table in a query. See the Apache wiki for information about setting this parameter. If this parameter is set, queries that access more than 1000 partitions fail with the following error: MetaException: Number of partitions scanned (=%d) on table '%s' exceeds limit (=%d) Setting this parameter protects against bad workloads and identifies queries that need to be optimized. To resolve the failed queries: Apply the appropriate partition filters. Override the limit on a per-query basis. Increase the cluster-wide limit beyond 1000, if needed, but note that this adds memory pressure to HiveServer2 and the Hive metastore.   ... View more

You can find the overview of CM internals in this blog https://blog.cloudera.com/how-does-cloudera-manager-work and the details of health tests on each service in https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ht.html ... View more

Thanks for the awesome explanation!! This comment from spark explains on the reason for allowing insecure connection https://issues.apache.org/jira/browse/SPARK-26019?focusedCommentId=16719231&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16719231  ... View more

May I know the exact steps you followed to replicate the issue? Are you noticing this error when running any code snippet. Can we have a shorter version of the script to replicate on my side and evaluate further? ... View more

time series graph of which metric are you looking for. HTTPFS has the following metrics https://docs.cloudera.com/documentation/enterprise/5-12-x/topics/cm_metrics_httpfs.html#cm_metrics_httpfs ... View more