Hive CLI error

[#1]

Hi experts,

I have been having trounble accessing the hiveCLI with tez and I found this in the application logs.

2021-03-15 09:28:10,781 [ERROR] [main] |app.DAGAppMaster|: Incompatible versions found, clientVersion=0.7.0.2.6.1.0-129, AMVersion=0.8.4.2.6.1.0-129

Can someone please tell me how to resolve this?

Thanks,

[#2]

@ryu 
How is your cluster setup?  The number of nodes and the HDP versions?

Are you running your HQL from the edge node?
 

Give as much information as possible.

[#3]

I have about 7 nodes in the cluster

HDP version 2.6.1.0-129

And I have recently deployed a new cluster.

When trying to access the hiveCLI, the "incompatible" error came from the application logs.

2021-03-15 11:40:46,401 [INFO] [main] |app.DAGAppMaster|: Comparing client version with AM version, clientVersion=0.7.0.2.6.1.0-129, AMVersion=0.8.4.2.6.1.0-129
2021-03-15 11:40:46,402 [ERROR] [main] |app.DAGAppMaster|: Incompatible versions found, clientVersion=0.7.0.2.6.1.0-129, AMVersion=0.8.4.2.6.1.0-129

And this is the error that I got when tried to run hive CLI in the terminal on one of the nodes in the cluster.

[root@test02 ~]# hive
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender.

Logging initialized using configuration in file:/etc/hive/2.6.1.0-129/0/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No cluster diagnostics found.
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:560)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:699)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:218)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:557)
... 8 more

[#4]

@ryu 
try the following solution. Always note all the changes you make incase you will need to revert 

Follow these steps to resolve the issue: 

1. Open Ambari.
2. Go to TEZ / Configs / Advanced tez-site.
3. Locate the configuration tez.history.logging.service.class.
4. Replace the value org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService with the new value: org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService"
5. Save the configuration changes.
6. Restart all services that Ambari is asking you to restart.
Then retry the 

[root@test02 ~]# hive

Please revert 

[#5]

Thanks @Shelton for the response.

I believe I had already tried this.

I tried to replace the ambari tez configuration again but still the same error.

Also our production cluster currently has the old value, but for some reason the production cluster is working fine.