Support Questions
Find answers, ask questions, and share your expertise

hive upgrade tool failed at hdp 3 upgrade

Solved Go to solution
Highlighted

hive upgrade tool failed at hdp 3 upgrade

New Contributor

I'm upgrading from hdp 2.6 to 3 The ambari upgrade from 2.6.2 to 2.7 went fine, I'm following the documentation, I'm at the step 4 on the page https://docs.hortonworks.com.s3.amazonaws.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/... I'm getting:

$JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar:/usr/hdp/2.6.0.3-8/hive2/lib/*:/usr/hdp/2.6.0.3-8/hadoop/*:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar: org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute                 SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hive2/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2018-07-26T13:34:30,051 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Starting with execute=true, location=.
2018-07-26T13:34:30,056 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Using Hive Version: 2.1.0.2.6.0.3-8 build: 2.1.0.2.6.0.3-8 from 21903e6a60f3f4f3de63731c0c7cbec2cb2e265e by jenkins source checksum 7af682f1761a67fa9e6f01d987cd6003
2018-07-26T13:34:30,094 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file null
2018-07-26T13:34:31,415 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-07-26T13:34:31,523 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Creating metastore client for PreUpgradeTool
2018-07-26T13:34:31,567 ERROR [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - PreUpgradeTool failed
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148) [hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082) ~[?:1.8.0_111]
        at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_111]
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        ... 6 more
Exception in thread "main" java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)


        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650)
        ... 6 more

I kinited as hive, running this as root. After some googling I found that having more than 1 version of org.apache.hadoop.hive.metastore.HiveMetaStoreClient might be an issue, so I did this:

# grep -r 'org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>' /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar  /usr/hdp/2.6.0.3-8/hive2/* /usr/hdp/2.6.0.3-8/hadoop/* /usr/hdp/2.6.0.3-8/hadoop-mapreduce/* /usr/hdp/2.6.0.3-8/hadoop-hdfs/* /usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/* /tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar /usr/hdp/2.6.0.3-8/hive/conf/*
[root@lsp7148c tmp]# grep -r 'org.apache.hadoop.hive.conf.HiveConf' /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar  /usr/hdp/2.6.0.3-8/hive2/* /usr/hdp/2.6.0.3-8/hadoop/* /usr/hdp/2.6.0.3-8/hadoop-mapreduce/* /usr/hdp/2.6.0.3-8/hadoop-hdfs/* /usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/* /tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar /usr/hdp/2.6.0.3-8/hive/conf/*
Binary file /usr/hdp/2.6.0.3-8/hive2/jdbc/hive-jdbc-2.1.0.2.6.0.3-8-standalone.jar matches
Binary file /usr/hdp/2.6.0.3-8/hive2/lib/hive-exec-2.1.0.2.6.0.3-8.jar matches
Binary file /usr/hdp/2.6.0.3-8/hive2/lib/hive-common-2.1.0.2.6.0.3-8.jar matches

I tried remove 2 out of 3 of these files but that didn't change anything.

Any idea on how to continue would be appreciated.

thanks in advance!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

I have the same issue. it seems to be ralated to HIVE-15081 (https://issues.apache.org/jira/browse/HIVE-15081) which was fixed in HDP 2.6.2. I am using HDP 2.6.1. Is there a way to apply this patch?

View solution in original post

12 REPLIES 12
Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

Explorer

Hi, I just had myself quite a few issues with this preupgrade tool, mainly from the fact that within the documentation there was the wrong classpath and I had to correct myself all of the wrong occurrences.

The preupgradetool needs the conf.server folder passed (/usr/hdp/current/hive-client/conf/conf.server), other than the other jars needed for execution. It looks like you haven't passed that argument in your classpath. Could you check if passing all of the correct folders/files in your execution command works?

My complete corrected classpath was the following:

$JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/current/hive-server2-hive2/lib/derby-10.10.2.0.jar:/usr/hdp/current/hive-server2-hive2/lib/*:/usr/hdp/current/hadoop/*:/usr/hdp/current/hadoop/lib/*:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/current/hadoop-mapreduce-client/lib/*:/usr/hdp/current/hadoop-hdfs/*:/usr/hdp/current/hadoop-hdfs/lib/*:/usr/hdp/current/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/current/hive-client/conf/conf.server:/usr/hdp/current/hive-metastore/lib/hive-metastore.jar:/usr/hdp/current/hive-metastore/lib/libthrift-0.9.3.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hive-client/lib/hive-common.jar:/usr/hdp/current/hive-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute
Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

Hi @Federico D'Ambrosio, Thanks for that. I actually believe that I did include the conf.server when I ran this but somehow didn't paste the right thing when creating this post. Sorry for the confusion. I also had issues with the libs but verified that all the ones I'm using exist on my side. Now, I also added what I saw in your command that I hadn't included earlier, but I'm still getting the same error

[root@host ~]# export JAVA_HOME=/opt/app/jdk1.8.0_111
[root@host ~]# $JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar:/usr/hdp/2.6.0.3-8/hive2/lib/*:/usr/hdp/2.6.0.3-8/hadoop/*:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/2.6.0.3-8/hive/conf/conf.server/*:/usr/hdp/2.6.0.3-8/hive/lib/hive-metastore.jar:/usr/hdp/2.6.0.3-8/hive/lib/libthrift-0.9.3.jar:/usr/hdp/2.6.0.3-8/hadoop/hadoop-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/hive-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/commons-cli-1.2.jar:/usr/hdp/2.6.0.3-8/hadoop* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hive2/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2018-07-31T12:33:48,044 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Starting with execute=true, location=.
2018-07-31T12:33:48,048 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Using Hive Version: 2.1.0.2.6.0.3-8 build: 2.1.0.2.6.0.3-8 from 21903e6a60f3f4f3de63731c0c7cbec2cb2e265e by jenkins source checksum 7af682f1761a67fa9e6f01d987cd6003
2018-07-31T12:33:48,074 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file null
2018-07-31T12:33:48,630 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-07-31T12:33:48,654 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Creating metastore client for PreUpgradeTool
2018-07-31T12:33:48,670 ERROR [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - PreUpgradeTool failed
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148) [hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082) ~[?:1.8.0_111]
        at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_111]
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        ... 6 more
Exception in thread "main" java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650)
        ... 6 more

If you have any additional tip please let me know

Re: hive upgrade tool failed at hdp 3 upgrade

Explorer

Hi @yves dormont, I believe that if you write in your classpath:

/usr/hdp/2.6.0.3-8/hive/conf/conf.server/*

with the wildcard '\*' you are actually including all of the files in the folder which have the extension *.jar. (Source)

So if you use

/usr/hdp/2.6.0.3-8/hive/conf/conf.server

you would be actually including all of the config files (properties and xml, not really sure which are used by the preupgrade tool, presumably hivemetastore-site.xml) within the conf.server folder, where all the information needed for the instantiation of the HiveMetaStoreClient.

Could you please change that part of your classpath and see if it changes anything?

Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

I'm facing the exact same issue as you but nothing is working. Tried many things today. If I find a resolution I will post it here.

Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

Hi @Federico D'Ambrosio,

I just tried that too, but still the same problem:

[root@lsp7148c ~]# export JAVA_HOME=/opt/app/jdk1.8.0_111
[root@lsp7148c ~]# kinit -kt /opt/app/hadoop-security/keytabs/hive.service.keytab hive/lsp7148c.sebank.se
[root@lsp7148c ~]# $JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/2.6.0.3-8/hive2/lib/derby-10.10.2.0.jar:/usr/hdp/2.6.0.3-8/hive2/lib/*:/usr/hdp/2.6.0.3-8/hadoop/*:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/2.6.0.3-8/hive/conf/conf.server:/usr/hdp/2.6.0.3-8/hive/lib/hive-metastore.jar:/usr/hdp/2.6.0.3-8/hive/lib/libthrift-0.9.3.jar:/usr/hdp/2.6.0.3-8/hadoop/hadoop-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/hive-common.jar:/usr/hdp/2.6.0.3-8/hive/lib/commons-cli-1.2.jar:/usr/hdp/2.6.0.3-8/hadoop* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hive2/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2018-08-01T09:20:08,278 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Starting with execute=true, location=.
2018-08-01T09:20:08,282 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Using Hive Version: 2.1.0.2.6.0.3-8 build: 2.1.0.2.6.0.3-8 from 21903e6a60f3f4f3de63731c0c7cbec2cb2e265e by jenkins source checksum 7af682f1761a67fa9e6f01d987cd6003
2018-08-01T09:20:08,309 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file file:/etc/hive/2.6.0.3-8/0/conf.server/hive-site.xml
2018-08-01T09:20:08,748 WARN [main] org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.enforce.sorting does not exist
2018-08-01T09:20:08,748 WARN [main] org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.enforce.bucketing does not exist
2018-08-01T09:20:08,869 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-08-01T09:20:08,893 INFO [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - Creating metastore client for PreUpgradeTool
2018-08-01T09:20:08,909 ERROR [main] org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool - PreUpgradeTool failed
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204) ~[hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148) [hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082) ~[?:1.8.0_111]
        at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_111]
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650) ~[hive-exec-2.1.0.2.6.0.3-8.jar:2.1.0.2.6.0.3-8]
        ... 6 more
Exception in thread "main" java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:89)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getHMS(PreUpgradeTool.java:190)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:204)
        at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:148)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1650)
        ... 6 more

On my side I'm looking more into the metastore now.

Will let you know if I get anywhere...

Thanks again!

Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

I have the same issue. it seems to be ralated to HIVE-15081 (https://issues.apache.org/jira/browse/HIVE-15081) which was fixed in HDP 2.6.2. I am using HDP 2.6.1. Is there a way to apply this patch?

Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

I have the same issue. it seems to be ralated to HIVE-15081 (https://issues.apache.org/jira/browse/HIVE-15081) which was fixed in HDP 2.6.2. I am using HDP 2.6.1. Is there a way to apply this patch?

View solution in original post

Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

I have made some test with the new Hive-libraries including HIVE-15081 and was able to connect to the metastore. To me it seems that everyone using Hive Version: 2.1.0.2.6.2.0-205 and above will be able to run the Hive-upgrade tool. for all users below this version it will fail.

Highlighted

Re: hive upgrade tool failed at hdp 3 upgrade

New Contributor

@Alexandre Lopes, can you confirm that you also started from a version below HDP 2.6.2?

@Federico D'Ambrosio, can you confirm that you started from HDP 2.6.2 or above?

Thanks a lot for that SG, good find!