question Re: hiveCLI not working with tez but works with mapreduce in Support Questions

hiveCLI not working with tez but works with mapreduce

ryu — Sun, 14 Mar 2021 20:04:46 GMT

Hi experts,

I am having issues accessing the hive CLI when using tez but it works with mapreduce.

I get the following error:

[root@test02 ~]# hive
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender.

Logging initialized using configuration in file:/etc/hive/2.6.1.0-129/0/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No cluster diagnostics found.
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:560)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:699)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:218)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:557)
... 8 more
[root@test02 ~]#

Any help is much appreciated.

Thanks,

Re: hiveCLI not working with tez but works with mapreduce

ChethanYM — Thu, 18 Mar 2021 01:01:58 GMT

Hi, This seems to be a generic error, can you upload complete HS2 logs where you can see these exceptions?

Is beeline works with Tez?

Re: hiveCLI not working with tez but works with mapreduce

ryu — Thu, 18 Mar 2021 15:37:11 GMT

Thanks @ChethanYM for the reply.

I did find this error in the application log

2021-03-18 10:33:27,963 [INFO] [main] |app.DAGAppMaster|: Comparing client version with AM version, clientVersion=0.7.0.2.6.1.0-129, AMVersion=0.8.4.2.6.1.0-129
2021-03-18 10:33:27,964 [ERROR] [main] |app.DAGAppMaster|: Incompatible versions found, clientVersion=0.7.0.2.6.1.0-129, AMVersion=0.8.4.2.6.1.0-129

Not exactly sure how to resolve this

Re: hiveCLI not working with tez but works with mapreduce

ryu — Thu, 18 Mar 2021 15:42:58 GMT

Sorry forgot to mention that this does work with beeline and I assume it is using tez because I have it set to tez in the hive.execution.engine.

The error above is in regards to hiveCLI only and in the application logs there seems to be some version mismatch which I am not exactly sure how to resolve.

Any help is much appreciated.

Thanks,

Re: hiveCLI not working with tez but works with mapreduce

mugdha — Fri, 19 Mar 2021 05:03:07 GMT

@ryu Are you able to connect to beeline? And can you make sure that the output of the below command has all components pointing to the current version in your cluster:

hdp-select

Here is the command help:

https://docs.cloudera.com/HDPDocuments/Ambari-2.1.2.1/bk_upgrading_Ambari/content/_Run_HDP_Select_mamiu.html

Re: hiveCLI not working with tez but works with mapreduce

isoardi — Fri, 19 Mar 2021 08:03:55 GMT

Hi,

I don't know which version of hive you are running, but hive-cli has been deprecated:

In HDP 3.0 and later, Hive does not support the following features:
Apache Hadoop Distributed Copy (DistCp)
WebHCat
Hcat CLI
Hive CLI (replaced by Beeline)
SQL Standard Authorization
MapReduce execution engine (replaced by Tez)

Re: hiveCLI not working with tez but works with mapreduce

ryu — Fri, 19 Mar 2021 12:48:24 GMT

Thanks @isoardi

We currently have hive 1.2.

Since hiveCLI is deprecated and as long as beeline or hiveserver2 works, then I should probably not bother fixing or worrying about hiveCLI right?
This is a sandbox cluster and using this cluster to prepare to eventually upgrade the production cluster.

Thanks,

Re: hiveCLI not working with tez but works with mapreduce

isoardi — Fri, 19 Mar 2021 13:49:23 GMT

Hi, yes it's right.

I don't recommend using the sandbox for testing of any upgrades of a production environment.
Many times the the sendbox env have been very sundry from the stnadard installed environments.

The sandbox is designed to understand the technology, but in your case I recommend having a test environment at scale compared to the production one, with the same versions and topology.

Re: hiveCLI not working with tez but works with mapreduce

ryu — Fri, 19 Mar 2021 15:12:22 GMT

@isoardi
oh this is a cluster that I can use to test upgrades etc.
But I did install HDP 2.6.x onto this cluster which is similar to our production environment.