Member since
05-26-2016
28
Posts
8
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2093 | 10-12-2017 09:18 PM | |
740 | 10-04-2017 11:37 AM |
10-23-2017
07:37 PM
@vishwa Can you also try with MR execution engine by setting below property as well: set hive.execution.engine=mr;
... View more
10-20-2017
01:44 AM
@vishwa Yes the explain plan should show a 'Sort Merge Join Operator'. In addition to the properties mentioned by @icocio you may also have to set below properties: set hive.input.format= org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.enforce.sortmergebucketmapjoin=false;
... View more
10-18-2017
01:39 AM
@Amith Dsilva - As mentioned by @Peter Kim the issue looks to be on HDFS side. RM is not able to find the /system/yarn/node-labels/nodelabel.mirror file on HDFS due to which its failing to start.
... View more
10-12-2017
09:23 PM
The Yarn application log should be able to provide more insight regarding the error. You can gather Yarn log using below command: yarn logs -applicationId application_1507606976106_0043
... View more
10-12-2017
09:23 PM
The Yarn application log should be able to provide more insight regarding the error. You can gather Yarn log using below command: yarn logs -applicationId application_1507606976106_0043
... View more
10-12-2017
09:18 PM
@viswanath From the stack looks like none of the LLAP daemons are running. Please check the LLAP application in Yarn RM UI and ensure you have LLAP daemon containers running. You can also check the HiveServer2 Interactive UI for this. java.lang.IllegalArgumentException: No running LLAP daemons! Please check LLAP service status and zookeeper
... View more
10-12-2017
11:36 AM
1 Kudo
@Saravanan Ramaraj Please refer to below link which gives some performance comparison between ORC and text: https://community.hortonworks.com/articles/69375/orc-columnar-format-vs-row-for-hive-tables.html
... View more
10-12-2017
05:07 AM
@Lakhman Pervatoju The reason for a MR job not being triggered while filtering a table is because for simple queries Hive does a direct fetch operation from HDFS which is much more faster/efficient than triggering a MR job. This is controlled by the property hive.fetch.task.conversion Below is a detailed description of this property: hive.fetch.task.conversion
Default Value: minimal in Hive 0.10.0 through 0.13.1, more in Hive 0.14.0 and laterAdded In: Hive 0.10.0 with HIVE-2925; default changed in Hive 0.14.0 with HIVE-7397Some select queries can be converted to a single FETCH task, minimizing latency. Currently the query should be single sourced not having any subquery and should not have any aggregations or distincts (which incur RS – ReduceSinkOperator, requiring a MapReduce task), lateral views and joins.
Supported values are none, minimal and more.
0. none: Disable hive.fetch.task.conversion (value added in Hive 0.14.0 with HIVE-8389)
1. minimal: SELECT *, FILTER on partition columns (WHERE and HAVING clauses), LIMIT only
2. more: SELECT, FILTER, LIMIT only (including TABLESAMPLE, virtual columns)
"more" can take any kind of expressions in the SELECT clause, including UDFs.
(UDTFs and lateral views are not yet supported – see HIVE-5718.)
... View more
10-12-2017
04:49 AM
@Robert Krimper Can you please provide complete error stack to better understand where the error may be coming from?
... View more
10-12-2017
04:43 AM
@Dinesh Chitlangia Since you have Atlas configured, you need to have Atlas Hook specified in hive.exec.post.hooks Here the issue looks to be due large number of queued tasks so you may want to check why there are so many queued tasks java.util.concurrent.ThreadPoolExecutor@5b868755[Running, pool size = 1, active threads = 1, queued tasks = 10000, completed tasks = 14807
... View more