Hello @Sadique Manzar! Sorry my huge delay, I've been busy these days. So, I took a look at your orc files and didn't see any orc file under the hdfs://hadoop/apps/hive/warehouse/sample.db/TABLEB. Do you mind to run the following command and share with me again? hdfs dfs -ls -R hdfs://hadoop/apps/hive/warehouse/sample.db/TABLEB/ Thanks. ... View more

Hi @Sadique Manzar! If your file doesn't have any sensitive data, then you can upload it here 🙂 Otherwise, just tell us if you have: More than 1 ORC file under the hdfs://hadoop/apps/hive/warehouse/sample.db/TABLEB If any of the ORC dumps has different typos What I am looking for is for any ORC with different datatype for a column under the same path. If so, this should explain why you can select with a limit and can't with a count or any grouping query. Hope this helps! ... View more

Hello @manjil subba! Sorry for the long delay. I've asked for those parameters, cause they seem a little bit higher than usual (that's probably the reason, why beeline is taking much longer than hiveCli). And also the vectorization set to false should impact the reduce process as well. You can adjust the TEZ performance by looking at this link below: https://community.hortonworks.com/articles/14309/demystify-tez-tuning-step-by-step.html And what you can do to check are both settings (hivecli/beeline) is: beeline -u 'jdbc:hive2://<HS2>:10000/default' -e "set;" > /tmp/beeline.properties hive -e "set;" > /tmp/hivecli.properties diff /tmp/beeline.properties /tmp/hivecli.properties BTW, in your beeline logs I didn't note the following message: Closing Tez Session Maybe we can ensure that all parameters are equal (beeline x hivecli) and after that, enable the debug level for beeline, to check what's going on. Hope this helps! 🙂 ... View more

Hi @manjil subba! Just asking, but, did you apply the same parameter for both (hiveCLI/beeline)? tez.am.resource.memory.mb=20000 mapreduce.map.memory.mb=20000 hive.vectorized.execution.reduce.enabled=false; And answering your question, for the job afaik, they should have the same behaviour. The only thing would be that beeline access the HS2/Thrift and hivecli does not. Hope this helps! ... View more

Hi @Sadique Manzar! I made a script to read/dump the ORC files under the hdfs path for your table. #!/bin/bash hdfs_orc_files=$(hdfs dfs -ls -R hdfs://hadoop/apps/hive/warehouse/sample.db/TABLEB/*) for i in $hdfs_orc_files do echo "Entry for the orc file $i" >> /tmp/comparing_orc_dump.txt echo "" hive --orcfiledump $i >> /tmp/comparing_orc_dump.txt echo "" done Do you mind to share with us the output from the /tmp/comparing_orc_dump.txt? I wanted to see if is there any long typo's in your ORC files. Hope this helps! ... View more

Hello @Michele Proverbio! Just asking but.. what happens if you create a simple table without the org.apache.hadoop.hive.druid.DruidStorageHandler? Does it happen the same issue? If so, what we can do is check for errors/warnings on the hivemetastore logs. Otherwise, I'd say to enable the DEBUG for the HS2 log. And try to figure out what's going on with your HM. Hope this helps! ... View more

Hmm that's interesting @Sadique Manzar. Could you share with us your table desc? desc formatted <TABLE_NAME>; ps: if you need to get away with this error, we can turn off the vectorization for Hive. and it should work.. yeah, it will be like a workaround 🙂 https://issues.apache.org/jira/browse/HIVE-9739 Otherwise, let's check this out further 🙂 Hope this helps! ... View more

Hmmm @Michele Proverbio, it seems you're using root as the connectionUserName. javax.jdo.option.ConnectionUserName=root Could you try to connect with the following command and rerun the DDL (create external table) beeline -u 'jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2' -n root I'm expecting to see errors, but a different error this time (permission to write on HDFS for root). Usually, when we set up the DB for HiveMetastore, we use the hive user for as the owner of the Connection. https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-administration/content/using_hive_with_mysql.html I'm not sure if this is related to your issue, but I think it's worth to try 🙂 Hope this helps! ... View more

Hello @Sadique Manzar! Could you try to add the following parameters before executing your query? SET hive.vectorized.execution.enabled=true; set hive.mapjoin.hybridgrace.hashtable=false; SET hive.vectorized.execution.mapjoin.native.enabled=false; set hive.cbo.enable=true; set hive.fetch.task.conversion=none; #If you're using MAPJOIN then set the parameters below SET hive.auto.convert.join=true; SET hive.auto.convert.join.noconditionaltask=true; SET hive.auto.convert.join.noconditionaltask.size=1000000000; set hive.exec.dynamic.partition.mode=nonstrict; Hope this helps! ... View more