@vishwa Can you also try with MR execution engine by setting below property as well: set hive.execution.engine=mr; ... View more

@vishwa Yes the explain plan should show a 'Sort Merge Join Operator'. In addition to the properties mentioned by @icocio you may also have to set below properties: set hive.input.format= org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; set hive.enforce.sortmergebucketmapjoin=false; ... View more

@Amith Dsilva - As mentioned by @Peter Kim the issue looks to be on HDFS side. RM is not able to find the /system/yarn/node-labels/nodelabel.mirror file on HDFS due to which its failing to start. ... View more

The Yarn application log should be able to provide more insight regarding the error. You can gather Yarn log using below command: yarn logs -applicationId application_1507606976106_0043 ... View more

The Yarn application log should be able to provide more insight regarding the error. You can gather Yarn log using below command: yarn logs -applicationId application_1507606976106_0043 ... View more

@Saravanan Ramaraj Please refer to below link which gives some performance comparison between ORC and text: https://community.hortonworks.com/articles/69375/orc-columnar-format-vs-row-for-hive-tables.html ... View more

@Lakhman Pervatoju The reason for a MR job not being triggered while filtering a table is because for simple queries Hive does a direct fetch operation from HDFS which is much more faster/efficient than triggering a MR job. This is controlled by the property hive.fetch.task.conversion Below is a detailed description of this property: hive.fetch.task.conversion Default Value: minimal in Hive 0.10.0 through 0.13.1, more in Hive 0.14.0 and laterAdded In: Hive 0.10.0 with HIVE-2925; default changed in Hive 0.14.0 with HIVE-7397Some select queries can be converted to a single FETCH task, minimizing latency. Currently the query should be single sourced not having any subquery and should not have any aggregations or distincts (which incur RS – ReduceSinkOperator, requiring a MapReduce task), lateral views and joins. Supported values are none, minimal and more. 0. none: Disable hive.fetch.task.conversion (value added in Hive 0.14.0 with HIVE-8389) 1. minimal: SELECT *, FILTER on partition columns (WHERE and HAVING clauses), LIMIT only 2. more: SELECT, FILTER, LIMIT only (including TABLESAMPLE, virtual columns) "more" can take any kind of expressions in the SELECT clause, including UDFs. (UDTFs and lateral views are not yet supported – see HIVE-5718.) ... View more

@Robert Krimper Can you please provide complete error stack to better understand where the error may be coming from? ... View more

@Dinesh Chitlangia Since you have Atlas configured, you need to have Atlas Hook specified in hive.exec.post.hooks Here the issue looks to be due large number of queued tasks so you may want to check why there are so many queued tasks java.util.concurrent.ThreadPoolExecutor@5b868755[Running, pool size = 1, active threads = 1, queued tasks = 10000, completed tasks = 14807 ... View more