Member since
12-11-2015
244
Posts
31
Kudos Received
32
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 338 | 07-22-2025 07:58 AM | |
| 947 | 01-02-2025 06:28 AM | |
| 1581 | 08-14-2024 06:24 AM | |
| 3115 | 10-02-2023 06:26 AM | |
| 2385 | 07-28-2023 06:28 AM |
02-27-2020
09:52 PM
This property is hive specific so it can be either enabled on hiveserver2 server level or in the client session level but is not possible to set on queue level (coz scheduler dont understand hive specific properties). If you set this property in hiveserver2 level - then all hive client will be mandated to use where clause when user run query on partitioned table. Other property which you can use is hive.metastore.limit.partition.request https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/admin_hive_tuning.html Quoting from link Set the hive.metastore.limit.partition.request parameter to 1000 to limit the maximum number of partitions accessed from a single table in a query. See the Apache wiki for information about setting this parameter. If this parameter is set, queries that access more than 1000 partitions fail with the following error: MetaException: Number of partitions scanned (=%d) on table '%s' exceeds limit (=%d) Setting this parameter protects against bad workloads and identifies queries that need to be optimized. To resolve the failed queries: Apply the appropriate partition filters. Override the limit on a per-query basis. Increase the cluster-wide limit beyond 1000, if needed, but note that this adds memory pressure to HiveServer2 and the Hive metastore.
... View more
02-27-2020
06:38 PM
Thanks @venkatsambath & @EricL this would certainly help in my hadoop journey. Regards
... View more
02-26-2020
05:08 AM
Thanks @venkatsambath That helped. Regards, Ansar
... View more
02-26-2020
12:15 AM
@venkatsambath Thanks for the assistance. I followed the same some time back and it worked, I had to add my ip address where postgresql is installed. That worked for me. Appreciate your help. Thanks. Ansar
... View more
02-21-2020
09:11 AM
Thanks for the awesome explanation!! This comment from spark explains on the reason for allowing insecure connection https://issues.apache.org/jira/browse/SPARK-26019?focusedCommentId=16719231&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16719231
... View more
02-20-2020
11:15 AM
@venkatsambath Thanks for your input's on this. I built the chart and your redirect provided extra metrics.
... View more
02-11-2020
10:34 PM
@venkatsambathIt's working thank you.
... View more
02-10-2020
06:45 AM
1 Kudo
You can pass that as command line argument Example: hbase org.apache.hadoop.hbase.mapreduce.RowCounter -Dmapreduce.job.cache.files=/test 'hbase_table_t10'
hbase org.apache.hadoop.hbase.mapreduce.RowCounter -Dmapreduce.job.cache.files=/test '<table_name>'
... View more
02-09-2020
07:19 PM
Sorry, I've not come across any scripts yet. For observability the cluster utilisation report is something that you can review to understand how weightage influenced the load. More details are in this link https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/admin_cluster_util_report.html#concept_edr_ntt_2v
... View more
04-08-2019
11:45 PM
1 Kudo
yes you have to upgrade to cdh6.1.0 or higher to use impala-3.1.0 - Its not possible to selectively upgrade impala alone https://www.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_61_packaging.html
... View more