Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.
abhishek_gupta
New Contributor

Re: Update and Inserts

by New Contributor in Support Questions
‎03-06-2019 07:07 PM
‎03-06-2019 07:07 PM
@Eugene Koifman We are facing an issue which seems to be a limitation of Hive 1.2 ACID tables. We are using MERGE for loading mutable data on Hive ACID tables but loading/Reading these ACID tables using Pig or using Spark seems to be an issue . Does Hive ACID table for Hive version 1.2 posses the capability of being read into Apache Pig using HCatLoader (or other means) or in Spark using SQLContext(or other means). For Spark, it seems it is only possible to read ACID tables if the table is fully compacted i.e no delta folders exist in any partition. Details in the following JIRA https://issues.apache.org/jira/browse/SPARK-15348, https://issues.apache.org/jira/browse/SPARK-15348 However I wanted to know if it is supported at all in Apache Pig to read ACID tables in Hive. When I tried reading both an un-partitoned/partitioned ACID table in Pig version 0.16 I get 0 records read. Successfully read 0 records from: "dwh.acid_table" HDP version 2.6.5 Spark version 2.3 Pig version 0.16 Hive version 1.2 ... View more

Re: Using Hive MERGE with Hive 1.2 on HDP 2.6.5

by New Contributor in Support Questions
‎01-21-2019 09:49 AM
‎01-21-2019 09:49 AM
Hi @Sindhu I am not able to get Hive2 Interactive to start on HDP 2.6.5, I have provided the details about my setup and issues I am facing in this post https://community.hortonworks.com/questions/231690/hive2-interactive-llap-fails-to-start.html Also, I did try the MERGE feature on Hive 1.2 after reading the comments on this blog https://hortonworks.com/blog/apache-hive-moving-beyond-analytics-offload-with-sql-merge/and it did work on Hive 1.2 ... View more

Using Hive MERGE with Hive 1.2 on HDP 2.6.5

by New Contributor in Support Questions
‎01-21-2019 06:37 AM
‎01-21-2019 06:37 AM
Can the MERGE DML feature added to Hive 1.2 on HDP 2.6.5 be used on Production. The following blog shows benchmarks that seem to be from Hive2 Interactive with LLAP https://hortonworks.com/blog/apache-hive-moving-beyond-analytics-offload-with-sql-merge/ However I wanted to understand if the same feature could be used with Hive 1.2, of course under performance limitations that are there with Hive 1.2. I also wanted to hear about use cases where the same is being used on Production. ... View more
Labels:

Re: Hive2 Interactive LLAP fails to start

by New Contributor in Support Questions
‎01-17-2019 04:04 PM
‎01-17-2019 04:04 PM
Yes, I did see this error when ambari issues a nohup nice on this script. However I am not sure why is this happening. Another error that is striking is the error by NetUtil.py failing to connect with Slide agent service as the OS Python version 2.7.6 where module ssl doesn’t have the attribute `_create_unverified_context`, this is resulting in the following error in slideragent log INFO 2019-01-17 09:04:59,889 NetUtil.py:66 - Failed to connect to https://shdp-ycn04:38097/ws/v1/slider/agents/ due to 'module' object has no attribute '_create_unverified_context' INFO 2019-01-17 09:04:59,889 NetUtil.py:85 - Server at https://shdp-ycn04:38097/ws/v1/slider/agents/ is not reachable, sleeping for 10 seconds... ... View more

Hive2 Interactive LLAP fails to start

by New Contributor in Support Questions
‎01-17-2019 02:56 PM
‎01-17-2019 02:56 PM
I have been trying to start Hive2 Interactive on HDP 2.6.5 that ships with Hive 2.1.0. I have followed the guidelines mentioned in the following articles about how to size and tune LLAP with Yarn https://community.hortonworks.com/articles/149486/llap-sizing-and-setup.html But LLAP daemon fails to start and Hive interactive gives up after configured retries. Also followed the following steps to troubleshoot and the error doesn't seem to be sizing related. https://community.hortonworks.com/articles/149899/investigating-when-llap-doesnt-start.html Following is my cluster setup in brief: hive.llap.daemon.yarn.container.mb= 10 GB llap_heap_size= 7 GB In-Memory Cache per Daemon= 2 GB Head room= 1 GB (hive.llap.daemon.yarn.container.mb should be = llap_heap_size + In-Memory Cache per Daemon + Headroom) Yarn container max= 14.6 GB, min 1 GB Number of executors per LLAP Daemon= 1 Yarn pre-emption enabled Retries for checking LLAP status= 20 Following are the startup script logs and Yarn container relevant logs. https://pastebin.com/raw/4ywJHp4M https://pastebin.com/raw/AiJkfwUR Following is a snapshot of Slide AM page Application: llap0 Status all containers allocated Total number of containers Create time: 17 Jan 2019 05:40:15 GMT Running since: 17 Jan 2019 05:40:15 GMT Time last flexed: N/A Application storage path: hdfs://hsft/user/hive/.slider/cluster/llap0/database Application configuration path: hdfs://hsft/user/hive/.slider/cluster/llap0/snapshot Component Instances Component Desired Actual Outstanding Requests Failed Failed to start Placement LLAP slider-appmaster Application Container Diagnostics Container ID Component State Exit Code Logs Diagnostics container_e27_1547703090681_0001_01_000002 LLAP 3 -1000 Logs container_e27_1547703090681_0001_01_000003 LLAP 3 -1000 Logs container_e27_1547703090681_0001_01_000004 LLAP 3 -1000 Logs container_e27_1547703090681_0001_01_000005 LLAP 3 -1000 Logs ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎03-12-2019 01:38 PM
Public Statistics
Date Registered ‎01-16-2019 11:46 AM
Last Visited ‎03-12-2019 01:38 PM
Total Messages Posted 5