Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

select query inside hive not working

Solved Go to solution

select query inside hive not working

Contributor

Hello Everyone

While triggering sql SELECT statement inside HIVE i get following error messages. I have mentioned the sql statement and the output below. Any suggestion will be highly appreciated.

 

hive (default)> show tables;
OK
order_items
Time taken: 0.35 seconds, Fetched: 1 row(s)
hive (default)> select count(1) from order_items;
Exception in thread "d413467f-6da8-4ebc-bf93-730e15b4b23f main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/io/HdfsUtils$HadoopFileStatus
    at org.apache.hadoop.hive.common.FileUtils.mkdir(FileUtils.java:545)
    at org.apache.hadoop.hive.ql.Context.getStagingDir(Context.java:237)
    at org.apache.hadoop.hive.ql.Context.getExtTmpPathRelTo(Context.java:429)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(SemanticAnalyzer.java:6437)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8961)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8850)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9703)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9596)
    at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:291)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10103)
    at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:228)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:239)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:473)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:319)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1249)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1295)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1178)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1166)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:236)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.io.HdfsUtils$HadoopFileStatus
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 30 more
[hduser@storage Softwares]$

1 ACCEPTED SOLUTION

Accepted Solutions

Re: select query inside hive not working

Contributor

I am able to solve the issue. I simply uninstall and re-install hive and with that it works. Select query is now able to show an output without any issue.

24 REPLIES 24

Re: select query inside hive not working

Guru

Can you please get the following info:

 

1. ps aux | grep HiveServer2

this is to capture the PID for HS2

 

2. strings /proc/<PID>/environ 

 

and paste the output of step 2 here for review. I would like to check what kind of JARs were loaded by HS2.

 

 

Re: select query inside hive not working

Contributor

Please find the output from below :

 

[hduser@storage Desktop]$ ps aux | grep HiveServer2
hduser    4479  0.0  0.0 103384   812 pts/0    S+   13:08   0:00 grep HiveServer2

 

[root@storage ~]# strings /proc/4479/environ
strings: '/proc/4479/environ': No such file

 

Re: select query inside hive not working

Guru
4479 was the PID for "grep", not for HS2. I can see that you were running it on your Desktop, can you please run this command on the HiveServer2 host instead?

Re: select query inside hive not working

Contributor

I am bit confused. Do you mean to run those command after i get login inside hive ?

[hduser@storage Desktop]$ jps
7041 NameNode
7891 NodeManager
7143 DataNode
7928 Jps
7291 SecondaryNameNode
7789 ResourceManager


[hduser@storage Desktop]$ hive
which: no hbase in (/usr/local/jdk1.8.0_111/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/jdk1.8.0_111/bin:/home/hduser/bin:/home/hduser/hadoop-2.6.5/bin:/home/hduser/pig-0.15.0/bin:/home/hduser/hadoop-2.6.5/sbin:/home/hduser/hadoop-2.6.5/bin:/home/hduser/hadoop-2.6.5/bin:/home/hduser/hadoop-2.6.5/sbin:/home/hduser/scala/:/bin/:/home/hduser/Softwares/sqoop/bin/:/home/hduser/Softwares/apache-hive-2.0.1-bin/bin/:/home/hduser/bin:/home/hduser/scala//bin/:/home/hduser/Softwares/sqoop//bin/:/usr/local/jdk1.8.0_111/bin:/home/hduser/bin:/home/hduser/hadoop-2.6.5/bin:/home/hduser/pig-0.15.0/bin:/home/hduser/hadoop-2.6.5/sbin:/home/hduser/hadoop-2.6.5/bin:/home/hduser/hadoop-2.6.5/bin:/home/hduser/hadoop-2.6.5/sbin:/home/hduser/scala/:/bin/:/home/hduser/Softwares/sqoop/bin/:/home/hduser/Softwares/apache-hive-2.0.1-bin/bin/)

Logging initialized using configuration in jar:file:/home/hduser/Softwares/apache-hive-2.0.1-bin/lib/hive-common-2.1.1.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> ps aux | grep HiveServer2 ;
NoViableAltException(26@[])
    at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1099)
    at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204)
    at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:440)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:319)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1249)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1295)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1178)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1166)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:236)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
FAILED: ParseException line 1:0 cannot recognize input near 'ps' 'aux' '|'

Re: select query inside hive not working

Guru
Hmm, sorry, I just noticed that you are running hive CLI, not using beeline, and you do not have HiveServer2 running in the cluster.

I will think further what else to check. Will update later on the thread.
Highlighted

Re: select query inside hive not working

Contributor

Thanks. Will look forward to hear from you soon.

Re: select query inside hive not working

Guru

Hi,

I have checked, class org.apache.hadoop.hive.io.HdfsUtils$HadoopFileStatus.class should be located under $HIVE_HOME/lib/hive-exec-2.2.0-SNAPSHOT.jar, the version number might be different, can you please make sure that it exist?

Did you install hadoop and hive tarballs manually?

Re: select query inside hive not working

Contributor

Nope i don't have that file inside lib folder. I am using apache-hive-2.0.1-bin version. So i downloaded tar.gz file of this version but i did not find the jar file which you have mentioned. So can you recomend me the url where i can download the file for apache-hive-2.0.1-bin version.

 

Yes i installed hadoop and hive manually.

Re: select query inside hive not working

Guru
I just downloaded the apache-hive-2.0.1-bin version and I can see it is under $HIVE_HOME/lib/hive-exec-2.0.1.jar

Have you setup $HIVE_HOME correctly? How did you install Hadoop and Hive? Where are the locations?