Member since
02-19-2020
2
Posts
0
Kudos Received
0
Solutions
02-27-2020
09:52 PM
This property is hive specific so it can be either enabled on hiveserver2 server level or in the client session level but is not possible to set on queue level (coz scheduler dont understand hive specific properties). If you set this property in hiveserver2 level - then all hive client will be mandated to use where clause when user run query on partitioned table. Other property which you can use is hive.metastore.limit.partition.request https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/admin_hive_tuning.html Quoting from link Set the hive.metastore.limit.partition.request parameter to 1000 to limit the maximum number of partitions accessed from a single table in a query. See the Apache wiki for information about setting this parameter. If this parameter is set, queries that access more than 1000 partitions fail with the following error: MetaException: Number of partitions scanned (=%d) on table '%s' exceeds limit (=%d) Setting this parameter protects against bad workloads and identifies queries that need to be optimized. To resolve the failed queries: Apply the appropriate partition filters. Override the limit on a per-query basis. Increase the cluster-wide limit beyond 1000, if needed, but note that this adds memory pressure to HiveServer2 and the Hive metastore.
... View more