Created on 12-13-2016 02:25 PM - edited 08-18-2019 05:48 AM
Hello
HDP 2.3.2 & Ambari 2.1.2.1 that was upgraded to Ambari 2.4.2.0
After upgrading Ambari to 2.4.2.0 when i start Hive service the operation output is red with an error - but Hive is green, up & running without any problems.
Here is the error:
But this is the actual result - the restart indicator still exists and Hive is green. I'm also able to connect to hive using beeline and work as usual.
Service Check produces the following result
The hiveserver2.log is free of errors and Hive is working properly ! I can live with this but i'm afraid that i will have problems with upgrading the HDP because one of the prerequisites is successful service check...
Any ideas ?
Created 12-13-2016 04:26 PM
Please run the below command:
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
.
And place the following symlink on all hosts (specially hive host)
# ls -l /usr/share/java/mysql-connector-java.jar lrwxrwxrwx 1 root root 31 Dec 2 07:48 /usr/share/java/mysql-connector-java.jar -> mysql-connector-java-5.1.17.jar
.
Looks like ambari is removing the mysql-connector jars symlink.
The changes that i suggested above to create the symlink is because it is not the "hiveserver2" failing but the "hive_service.py" is failing. Which is looking for the mysql jar.
Created 12-13-2016 02:33 PM
From your 3rd error screen shot it looks like the "zookeeper.jar" path is set incorrectly.
Please correct the "templeton.libjars" Property value as following:
/usr/hdp/${hdp.version}/zookeeper/zookeeper.jar,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar
- In the image we see that the zookeeper.jar path is incorrect.
- The "templeton.libjars" can be set via ambari
"Hive" ==> "config" ==> "Advanced" ==> "Advanced webhcat-site"
Created 12-13-2016 02:45 PM
Hi @jss
Thank you for your reply
The "templeton.libjars" Property hasen't been changed so i'm kind of baffled why i need to "correct" it. I do agree that the path which the service check is looking seems corrupt:
/usr/hdp/2.3.2.0-2950/hive/lib/hive-common.jar/zookeeper.jar
hive-common.jar/zookeeper.jar (jar file \ jar file)
It seems like a bug in the service check.... there can't be a .jar file under a .jar file... Or am i missing here something ?
Created 12-13-2016 02:50 PM
You encountered a bug regarding the "templeton.libjars" Property. Please file it. Also please correct that "templeton.libjars" Property value and the service check issue should be fixed.
/usr/hdp/${hdp.version}/zookeeper/zookeeper.jar,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar
- Else please compare the value of "templeton.libjars" property from any of your working environment and you will find that it will be same as above.
Created 12-13-2016 02:54 PM
I've compared to other working environments and the value is the same in all of them:
/usr/hdp/${hdp.version}/zookeeper,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar/zookeeper.jar
Created 12-13-2016 02:44 PM
It seems Hive Metastore is having an issue with hive DB.
might be with following reason:
The HIVE metastore is a different host to the host running HiveServer2. Permissions on the metastore do not allow connections from the HiveServer2 host.
Created 12-13-2016 02:50 PM
Thank you for replying!
The HIVE metastore is on the same host as the running HiveServer2. Nothing was changed in the hive mysql \ metastore- only Ambari was upgraded. That's the only change. Also - if there was a problem with connection between the HS2 to the metastore - Hive wouldn't work, but fortunately it works without a problem!
I have NO problems with restarting Hive nor working with it. I can connect to any Hive DBs, query as usual. Permissions are working. Everything seems normal and green. My problem is that the restart indicator doesn't go away + restarting produces error although it succeeds with restarting + service check fails.
Created 12-13-2016 03:06 PM
just made small modification in any of your config file and give a try to re-start that service
Created 12-13-2016 02:56 PM
in your error logs , it says that the com.mysql.jdbc.driver could not be located. can you see if the mysql jdbc jar is in the location used by the service check. As per logs it should be at /usr/hdp/current/hive-server2/lib/mysql-connector-java.jar.
Created 12-13-2016 03:03 PM
So in general there seems to be two issues (both need to be addressed)
1. Incorrect "templeton.libjars"
>> Please correct the "templeton.libjars" Property value as following:
/usr/hdp/${hdp.version}/zookeeper/zookeeper.jar,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar
2. mysql connection JDBC jar is missing >> pelase check if the JAR is present or nor? Or If missing then you can copy it from other installation/host.
.