CDH6 installation - Failed to Validate Hive Metastore schema.

[#1]

Hi,

I am trying to install CDH 6 (I think I chose "Essentials" services package with Hive) but the "Run a set of services for the first time" fails on "Validating the Hive Metastore schema". 

I installed MariaDB and created databases manually as stated in the documentation and here is the error I get:

++ '[' '!' -f /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/lib/hive/lib/hive-cli-2.1.1-cdh6.0.0.jar ']'
++ false
++ exec /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/lib/hadoop/bin/hadoop jar /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/lib/hive/lib/hive-cli-2.1.1-cdh6.0.0.jar org.apache.hive.beeline.HiveSchemaTool -verbose -validate -dbType mysql
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to retrieve schema tables from Hive Metastore DB,Not supported
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to retrieve schema tables from Hive Metastore DB,Not supported
	at org.apache.hive.beeline.HiveSchemaTool.validateSchemaTables(HiveSchemaTool.java:739)
	at org.apache.hive.beeline.HiveSchemaTool.doValidate(HiveSchemaTool.java:590)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1142)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
*** schemaTool failed ***

Thanks for any help.

[#2]

Please confirm what version of mysql-connector-java.jar are you using? Can you try to upgrade to latest version if not the latest yet?

[#3]

Hi,

I have the same problem with CDH 6 and the last version of mysql-connector-java-8.0.13.1.el7.

I change to previous version :mysql-connector-java-5.1.47 then reload installation: all fine.

Hope this help.

[#4]

Hi ,

I am facing same situation i am using

[root@quickstart ~]# rpm -qa|grep mysql-connector-java
mysql-connector-java-5.1.25-3.el7.noarch

kindly guide me the steps to get rid of the below error in CDH 6.

++ shift
++ '[' -verbose == 'hive-cli-*.jar' ']'
++ '[' -verbose == 'hive-beeline-*.jar' ']'
++ '[' true == true ']'
++ JAR='hive-cli-*.jar'
++ '[' '!' -f /opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hive/lib/hive-cli-2.1.1-cdh6.0.1.jar ']'
++ false
++ exec /opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/bin/hadoop jar /opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hive/lib/hive-cli-2.1.1-cdh6.0.1.jar org.apache.hive.beeline.HiveSchemaTool -verbose -validate -dbType mysql
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to retrieve schema tables from Hive Metastore DB,Not supported
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to retrieve schema tables from Hive Metastore DB,Not supported
	at org.apache.hive.beeline.HiveSchemaTool.validateSchemaTables(HiveSchemaTool.java:739)
	at org.apache.hive.beeline.HiveSchemaTool.doValidate(HiveSchemaTool.java:590)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1142)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
*** schemaTool failed ***

if you are mentioning that "mysql-connector-java-5.1.47" will resolve the issue.kindly provide the RPM link for the same. or if need to install the tar.gz file kindly guide steps to do it on CentOS 7.6

Regards

Nitesh

[#5]

Hi,

You will find all your needs at this link:

topic: https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5...

Hope this help you !

[#6]

Hi Alex , Many many thanks for the Quick Reply. i followed below steps [root@quickstart ~]# mkdir -p /usr/share/java/ [root@quickstart ~]# systemctl stop mysqld [root@quickstart ~]# yum remove mysql-connector-java-5.1.25-3.el7.noarch [root@quickstart ~]# cd mysql-connector-java-5.1.46 [root@quickstart mysql-connector-java-5.1.46]# ls build.xml CHANGES COPYING mysql-connector-java-5.1.46-bin.jar mysql-connector-java-5.1.46.jar README README.txt src [root@quickstart mysql-connector-java-5.1.46]# cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar [root@quickstart mysql-connector-java-5.1.46]# systemctl start mysqld Then start the cluster. Regards Nitesh

[#7]

I also just ran into this issue.

The way I solved it was to install the latest MySQL JDBC Driver as described here:

https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5_3

This action must be performed on the node(s) with the Hive Metastore role installed.

[#8]

I did a quick research on why rhel7/centos7 still contains such an old mysql java connector/driver version (see "Redhat response"), and checked for working alternative RPMs (as installing via an rpm has many advantages vs a manual tarball install)

There are many working RPMs (I tested all of below, with CDH 6.2.0 and openjdk-8):

- With Java >=v8, I recommend following:
Ref: https://centos.pkgs.org/7/mysql-connectors-i386/mysql-connector-java-8.0.16-1.el7.noarch.rpm.html
RPM: yum install http://repo.mysql.com/yum/mysql-connectors-community/el/7/i386//mysql-connector-java-8.0.16-1.el7.no...

- With Java v7 you can try the latest 5.1.x from Fedora:
Ref: https://fedora.pkgs.org/29/fedora-x86_64/mysql-connector-java-5.1.38-7.fc29.noarch.rpm.html
RPM: http://download-ib01.fedoraproject.org/pub/fedora/linux/releases/29/Everything/x86_64/os/Packages/m/...

- Find RPMs for other distros: https://pkgs.org/download/mysql-connector-java

- Official Redhat response:
Alternatively, for Java-8, Redhat proposes to use the MariaDB client: https://bugzilla.redhat.com/show_bug.cgi?id=1684349#c7
> this is the best we could do for the customers who need a recent version of the JDBC driver for MySQL/MariaDB.

More infos: https://developers.redhat.com/blog/2019/06/25/mariadb-10-3-now-available-on-red-hat-enterprise-linux...
For centos7: https://centos.pkgs.org/7/centos-sclo-rh-testing-x86_64/rh-mariadb103-mariadb-java-client-2.4.1-1.el...

I tested the 1-line yum install, but CDH would require more changes, due to the driver installed in /opt:
yum install https://buildlogs.centos.org/centos/7/sclo/x86_64/rh/rh-mariadb103/rh-mariadb103-mariadb-java-client... https://buildlogs.centos.org/centos/7/sclo/x86_64/rh/rh-mariadb103/rh-mariadb103-runtime-3.3-3.el7.x...