Member since
02-02-2021
116
Posts
2
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
745 | 08-13-2021 09:44 AM | |
3693 | 04-27-2021 04:23 PM | |
1369 | 04-26-2021 10:47 AM | |
923 | 03-29-2021 06:01 PM | |
2746 | 03-17-2021 04:53 PM |
06-03-2021
03:54 PM
Also Squirrel seems to be connecting to the dev cluster. It just times out when running a query such as "show databases". If squirrel stays connected for a long time, I noticed that the query will eventually return results instead of timing out. Per cloudera "https://docs.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hive_metastore_configure.html#concept_jsw_bnc_rp" It says that minimum 4 dedicated cores to HS2 and 4 for hive metastore. The server that hosts hs2 and metastore only has a total of 8 cores. Can this be a reason for the performance issue? Any help on this is much appreciated. Thanks,
... View more
05-13-2021
07:28 PM
Hello Please check the below instructions (HDP platform) and make sure you are following the right steps https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/fault-tolerance/content/update_the_hive_metastore.html If you are using pure open source, you can seek helps from Hive community forum
... View more
05-12-2021
10:26 PM
Hi, You can go to this below link: https://docs.cloudera.com/documentation/enterprise/6/6.2.html Choose the CDH version you need to check -> Then choose "Release Notes" for more details. Below is the CDP "Release Notes: https://docs.cloudera.com/cdp-private-cloud/latest/release-guide/topics/cdpdc-release-notes-links.html
... View more
04-27-2021
04:23 PM
1 Kudo
Here is an update. So I finally was able to use ambari to get Hive installed and services started using the apache bigtop repo. Also was able to connect via hiveCLI as well as beeline(hiveserver2) and ran a simple "show databases;" which ran successfully. So after symlinking the following directories, hiveserver2 finally was able to start successfully. [root@test ~]# ll /usr/bgtp/current/ total 32 lrwxrwxrwx 1 root root 13 Apr 23 19:38 hive-client -> /usr/lib/hive lrwxrwxrwx 1 root root 13 Apr 23 19:37 hive-metastore -> /usr/lib/hive lrwxrwxrwx 1 root root 13 Apr 27 16:28 hive-server2 -> /usr/lib/hive lrwxrwxrwx 1 root root 22 Apr 27 16:29 hive-webhcat -> /usr/lib/hive-hcatalog I did not find any documentation on how to install hive with the apache bigtop packages using ambari. There was some documentation on how to install hive with the apache bigtop package using the command line. If anyone finds any documentation on how to install the different components in the apache bigtop package, please let me know. Thanks,
... View more
04-26-2021
10:47 AM
ok so I finally figured out how to have ambari run the command /usr/bin/yum -d 0 -e 0 -y install hive-hcatalog I had to modify the file: /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml under the <osSpecifics> tag. Restarted ambari and then ambari was able to run the correct command to have hive installed.
... View more
04-20-2021
10:53 AM
Hi @ryu , We can trigger manual GC in datanode JVM application AFAIK. The best way to deal with long GC pauses is to allocate right amount of heap memory. We recommend the formula of - 1gb heap per 1 million blocks. You can get the number of block count from NameNode Webui -> datanode (through ambari or CM). Increase the heap and that should fix your issue. Do check for "No GC detected" in Datanode logs, if you see those then it could be hardware problem triggering GC. Does that answers your questions. Let me know Regards, Vipin
... View more
04-13-2021
11:41 PM
Hello @ryu Please refer below doc for query failed with this error Result fetch timed out in hive view. https://community.cloudera.com/t5/Community-Articles/How-to-fix-ambari-Hive-View-1-5-2-0-Result-fetch-timed-out/ta-p/244965
... View more
04-10-2021
12:53 AM
1 Kudo
Hi @ryu, then you might need to build some customize in-house monitoring scripts using Yarn APIs or other tools like Prometheus or Grafana for your use case. Please also refer to the below links for more insights https://www.programmersought.com/article/61565532790/ http://rokroskar.github.io/monitoring-spark-on-hadoop-with-prometheus-and-grafana.html https://www.linkedin.com/pulse/how-monitor-yarn-application-via-restful-api-wayne-zhu/
... View more
04-09-2021
06:36 PM
Hello Have you tried to tune the performance? https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.6.5/bk_command-line-installation/content/tune_llap.html
... View more