question Re: hive upgrade tool failed at hdp 3 upgrade in Archives of Support Questions (Read Only)

hive upgrade tool failed at hdp 3 upgrade

yves_d — Thu, 26 Jul 2018 21:11:13 GMT

I'm upgrading from hdp 2.6 to 3 The ambari upgrade from 2.6.2 to 2.7 went fine, I'm following the documentation, I'm at the step 4 on the page https://docs.hortonworks.com.s3.amazonaws.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/prepare_hive_for_upgrade.html I'm getting:

$JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar:/usr/hdp/2.6.0.3-8/hive2/lib/*:/usr/hdp/2.6.0.3-8/hadoop/*:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar: org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute                 SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hive2/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2018-07-26T13:34:30,051 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Starting with execute=true, location=.
2018-07-26T13:34:30,056 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Using Hive Version: 2.1.0.2.6.0.3-8 build: 2.1.0.2.6.0.3-8 from 21903e6a60f3f4f3de63731c0c7cbec2cb2e265e by jenkins source checksum 7af682f1761a67fa9e6f01d987cd6003
2018-07-26T13:34:30,094 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file null
2018-07-26T13:34:31,415 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-07-26T13:34:31,523 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Creating metastore client for PreUpgradeTool
2018-07-26T13:34:31,567 ERROR [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - PreUpgradeTool failed
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148) [hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082) ~[?:1.8.0_111]
        at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_111]
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        ... 6 more
Exception in thread "main" java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)


        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650)
        ... 6 more

I kinited as hive, running this as root. After some googling I found that having more than 1 version of org.apache.hadoop.hive.metastore.HiveMetaStoreClient might be an issue, so I did this:

# grep -r 'org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>' /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar  /usr/hdp/2.6.0.3-8/hive2/* /usr/hdp/2.6.0.3-8/hadoop/* /usr/hdp/2.6.0.3-8/hadoop-mapreduce/* /usr/hdp/2.6.0.3-8/hadoop-hdfs/* /usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/* /tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar /usr/hdp/2.6.0.3-8/hive/conf/*
[root@lsp7148c tmp]# grep -r 'org.apache.hadoop.hive.conf.HiveConf' /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar  /usr/hdp/2.6.0.3-8/hive2/* /usr/hdp/2.6.0.3-8/hadoop/* /usr/hdp/2.6.0.3-8/hadoop-mapreduce/* /usr/hdp/2.6.0.3-8/hadoop-hdfs/* /usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/* /tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar /usr/hdp/2.6.0.3-8/hive/conf/*
Binary file /usr/hdp/2.6.0.3-8/hive2/jdbc/hive-jdbc-2.1.0.2.6.0.3-8-standalone.jar matches
Binary file /usr/hdp/2.6.0.3-8/hive2/lib/hive-exec-2.1.0.2.6.0.3-8.jar matches
Binary file /usr/hdp/2.6.0.3-8/hive2/lib/hive-common-2.1.0.2.6.0.3-8.jar matches

I tried remove 2 out of 3 of these files but that didn't change anything.

Any idea on how to continue would be appreciated.

thanks in advance!

Re: hive upgrade tool failed at hdp 3 upgrade

f_dambrosio — Thu, 26 Jul 2018 22:10:55 GMT

Hi, I just had myself quite a few issues with this preupgrade tool, mainly from the fact that within the documentation there was the wrong classpath and I had to correct myself all of the wrong occurrences.

The preupgradetool needs the conf.server folder passed (/usr/hdp/current/hive-client/conf/conf.server), other than the other jars needed for execution. It looks like you haven't passed that argument in your classpath. Could you check if passing all of the correct folders/files in your execution command works?

My complete corrected classpath was the following:

$JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/current/hive-server2-hive2/lib/derby-10.10.2.0.jar:/usr/hdp/current/hive-server2-hive2/lib/*:/usr/hdp/current/hadoop/*:/usr/hdp/current/hadoop/lib/*:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/current/hadoop-mapreduce-client/lib/*:/usr/hdp/current/hadoop-hdfs/*:/usr/hdp/current/hadoop-hdfs/lib/*:/usr/hdp/current/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/current/hive-client/conf/conf.server:/usr/hdp/current/hive-metastore/lib/hive-metastore.jar:/usr/hdp/current/hive-metastore/lib/libthrift-0.9.3.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hive-client/lib/hive-common.jar:/usr/hdp/current/hive-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute

Re: hive upgrade tool failed at hdp 3 upgrade

yves_d — Tue, 31 Jul 2018 17:35:40 GMT

Hi @Federico D'Ambrosio, Thanks for that. I actually believe that I did include the conf.server when I ran this but somehow didn't paste the right thing when creating this post. Sorry for the confusion. I also had issues with the libs but verified that all the ones I'm using exist on my side. Now, I also added what I saw in your command that I hadn't included earlier, but I'm still getting the same error

[root@host ~]# export JAVA_HOME=/opt/app/jdk1.8.0_111
[root@host ~]# $JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar:/usr/hdp/2.6.0.3-8/hive2/lib/*:/usr/hdp/2.6.0.3-8/hadoop/*:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/2.6.0.3-8/hive/conf/conf.server/*:/usr/hdp/2.6.0.3-8/hive/lib/hive-metastore.jar:/usr/hdp/2.6.0.3-8/hive/lib/libthrift-0.9.3.jar:/usr/hdp/2.6.0.3-8/hadoop/hadoop-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/hive-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/commons-cli-1.2.jar:/usr/hdp/2.6.0.3-8/hadoop* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hive2/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2018-07-31T12:33:48,044 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Starting with execute=true, location=.
2018-07-31T12:33:48,048 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Using Hive Version: 2.1.0.2.6.0.3-8 build: 2.1.0.2.6.0.3-8 from 21903e6a60f3f4f3de63731c0c7cbec2cb2e265e by jenkins source checksum 7af682f1761a67fa9e6f01d987cd6003
2018-07-31T12:33:48,074 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file null
2018-07-31T12:33:48,630 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-07-31T12:33:48,654 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Creating metastore client for PreUpgradeTool
2018-07-31T12:33:48,670 ERROR [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - PreUpgradeTool failed
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148) [hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082) ~[?:1.8.0_111]
        at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_111]
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        ... 6 more
Exception in thread "main" java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650)
        ... 6 more

If you have any additional tip please let me know

Re: hive upgrade tool failed at hdp 3 upgrade

f_dambrosio — Tue, 31 Jul 2018 22:15:15 GMT

Hi @yves dormont, I believe that if you write in your classpath:

/usr/hdp/2.6.0.3-8/hive/conf/conf.server/*

with the wildcard '\*' you are actually including all of the files in the folder which have the extension *.jar. (Source)

So if you use

/usr/hdp/2.6.0.3-8/hive/conf/conf.server

you would be actually including all of the config files (properties and xml, not really sure which are used by the preupgrade tool, presumably hivemetastore-site.xml) within the conf.server folder, where all the information needed for the instantiation of the HiveMetaStoreClient.

Could you please change that part of your classpath and see if it changes anything?

Re: hive upgrade tool failed at hdp 3 upgrade

alexrlops — Wed, 01 Aug 2018 03:39:03 GMT

I'm facing the exact same issue as you but nothing is working. Tried many things today. If I find a resolution I will post it here.

Re: hive upgrade tool failed at hdp 3 upgrade

yves_d — Wed, 01 Aug 2018 14:27:00 GMT

Hi @Federico D'Ambrosio,

I just tried that too, but still the same problem:

[root@lsp7148c ~]# export JAVA_HOME=/opt/app/jdk1.8.0_111
[root@lsp7148c ~]# kinit -kt /opt/app/hadoop-security/keytabs/hive.service.keytab hive/lsp7148c.sebank.se
[root@lsp7148c ~]# $JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar:/usr/hdp/2.6.0.3-8/hive2/lib/*:/usr/hdp/2.6.0.3-8/hadoop/*:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/2.6.0.3-8/hive/conf/conf.server:/usr/hdp/2.6.0.3-8/hive/lib/hive-metastore.jar:/usr/hdp/2.6.0.3-8/hive/lib/libthrift-0.9.3.jar:/usr/hdp/2.6.0.3-8/hadoop/hadoop-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/hive-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/commons-cli-1.2.jar:/usr/hdp/2.6.0.3-8/hadoop* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hive2/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2018-08-01T09:20:08,278 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Starting with execute=true, location=.
2018-08-01T09:20:08,282 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Using Hive Version: 2.1.0.2.6.0.3-8 build: 2.1.0.2.6.0.3-8 from 21903e6a60f3f4f3de63731c0c7cbec2cb2e265e by jenkins source checksum 7af682f1761a67fa9e6f01d987cd6003
2018-08-01T09:20:08,309 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file file:/etc/hive/2.6.0.3-8/0/conf.server/hive-site.xml
2018-08-01T09:20:08,748 WARN [main] org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.enforce.sorting does not exist
2018-08-01T09:20:08,748 WARN [main] org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.enforce.bucketing does not exist
2018-08-01T09:20:08,869 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-08-01T09:20:08,893 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Creating metastore client for PreUpgradeTool
2018-08-01T09:20:08,909 ERROR [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - PreUpgradeTool failed
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148) [hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082) ~[?:1.8.0_111]
        at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_111]
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        ... 6 more
Exception in thread "main" java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650)
        ... 6 more

On my side I'm looking more into the metastore now.

Will let you know if I get anywhere...

Thanks again!

Re: hive upgrade tool failed at hdp 3 upgrade

MundsonFarrell — Thu, 02 Aug 2018 21:12:22 GMT

I have the same issue. it seems to be ralated to HIVE-15081 (https://issues.apache.org/jira/browse/HIVE-15081) which was fixed in HDP 2.6.2. I am using HDP 2.6.1. Is there a way to apply this patch?

Re: hive upgrade tool failed at hdp 3 upgrade

MundsonFarrell — Thu, 02 Aug 2018 21:12:22 GMT

Re: hive upgrade tool failed at hdp 3 upgrade

MundsonFarrell — Thu, 02 Aug 2018 23:12:36 GMT

I have made some test with the new Hive-libraries including HIVE-15081 and was able to connect to the metastore. To me it seems that everyone using Hive Version: 2.1.0.2.6.2.0-205 and above will be able to run the Hive-upgrade tool. for all users below this version it will fail.

Re: hive upgrade tool failed at hdp 3 upgrade

yves_d — Fri, 03 Aug 2018 14:52:22 GMT

@Alexandre Lopes, can you confirm that you also started from a version below HDP 2.6.2?

@Federico D'Ambrosio, can you confirm that you started from HDP 2.6.2 or above?

Thanks a lot for that SG, good find!

Re: hive upgrade tool failed at hdp 3 upgrade

alexrlops — Fri, 03 Aug 2018 20:56:05 GMT

Unfortunately, I cannot confirm it. I had the chance and deleted my HDP 2.6.x installation and reinstalled the 3.0.0. Sorry.

Re: hive upgrade tool failed at hdp 3 upgrade

f_dambrosio — Mon, 20 Aug 2018 18:48:31 GMT

Hi, yes, I upgraded from HDP 2.6.5

Re: hive upgrade tool failed at hdp 3 upgrade

MundsonFarrell — Wed, 19 Sep 2018 17:36:34 GMT

In the meantime Hortonworks has reflected this limitation in the upgrade documentation:

Important
Ambari 2.7 only supports fully managing a HDP 3.0 cluster. If you are running any other HDP version, you must first upgrade to HDP 2.6 using Ambari 2.6. You must first upgrade to HDP 2.6.5 if you use Hive. Then, upgrade to Ambari 2.7 and use it to upgrade your cluster to HDP 3.0." https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/ambari_upgrade_guide.html

Important

Ambari 2.7 only supports fully managing a HDP 3.0 cluster. If you are running any other HDP version, you must first upgrade to HDP 2.6 using Ambari 2.6. You must first upgrade to HDP 2.6.5 if you use Hive. Then, upgrade to Ambari 2.7 and use it to upgrade your cluster to HDP 3.0."

https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/ambari_upgrade_guide.html