Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hi i met issue when upgrading hive from 0.10 to 0.11.

hi i met issue when upgrading hive from 0.10 to 0.11.

Contributor

hi hi i met an issue when upgrading hive from 0.10 to 0.11. what i have done for the upgrade withour cloudera manager:

 

1,download the hive related rpm packages, including hive-hbase,hive-jdbc,hive, hive-server2 and the dependcies avro-libs, parquet.

2, with command rpm -uvh to upgrade hive derectly with no error

3, upgrade metastore schema with upgrade-0.10.0-to-0.11.0.mysql.sql. everything shows okay.

 

then when i start hive, it seems still use the hive 0.10. i checked the version with command "hive --version", it shows Hive 0.11.0-cdh5.0.0-beta-1. but when i go to hive command interface and show functions, i cant see the function nvl and rank which should be in hive 0.11 functions repository.

 

and to troubleshooting the problem, i found there are hive0.10 jars in /usr/lib/hadoop/lib/. but basiclly we only set hive_home to /usr/lib/hive/lib/.

 

so i'm wondering is there anything wrong with my upgrade steps? and whats the use of the jars /usr/lib/hadoop/lib/? and how to fix this issue?

 

thanks in advance.

5 REPLIES 5

Re: hi i met issue when upgrading hive from 0.10 to 0.11.

Cloudera normally expects customers to upgrade all packages to a new
release instead of just selected components. So using Hive from a newer CDH
release with HDFS from an older release is not recommended.

Are you able to tell us the older and newer versions of CDH? Did you follow
an upgrade guide and if so could you provide me the URL?
Regards,
Gautam Gopalakrishnan

Re: hi i met issue when upgrading hive from 0.10 to 0.11.

Contributor

the older version is cdh4.4 and download the hive 0.11 from cdh5.0 b1 package repository. and following this upgrade guide https://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/5.0/CDH5-Installation-Guide/cdh...

i upgraded hive rpm package manaully(our network is not allowed to connnect to internet detectly). 

 

and another thing is i found there are the hive0.10 jars in /usr/lib/hadoop/lib, while the actual hive0.11 jars are located in /usr/lib/hive/lib/ and hive_home is /usr/lib/hive. does it matter with my problem?

 

thanks!

Re: hi i met issue when upgrading hive from 0.10 to 0.11.

Please try to not use beta releases when the final release is already
available. The latest release of CDH is 5.1.3 and it came out just today.
Please read the upgrade guide and upgrade *all* components, not just Hive.

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Release-Notes/CDH5-R...

​If you use Cloudera Manager, remember to upgrade that before CDH​
​ as CM4 cannot manage CDH5​


Regards,
Gautam Gopalakrishnan

Re: hi i met issue when upgrading hive from 0.10 to 0.11.

Contributor

actually we are not running cloudera manager. due to some enviroment limited, its not easy and not possible to upgrade the whole structure.

 

could you help do me a favor to troubleshoot my problem?

 

right now i made some test. i copied the hive0.11 jars from /usr/lib/hive/lib/ to /usr/lib/hadoop/lib/, and with command line hive, i could get those new functions. but when i start up hive-server2, there is an error saying 

 

2014-09-22 16:41:41,863 FATAL server.HiveServer2 (HiveServer2.java:main(94)) - Error starting HiveServer2

org.apache.hive.service.ServiceException: Failed to Start HiveServer2

        at org.apache.hive.service.CompositeService.start(CompositeService.java:80)

        at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:61)

        at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:92)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

Caused by: org.apache.hive.service.ServiceException: Unable to connect to MetaStore!

        at org.apache.hive.service.cli.CLIService.start(CLIService.java:94)

        at org.apache.hive.service.CompositeService.start(CompositeService.java:70)

        ... 7 more

Caused by: java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.

        at com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)

        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:30108)

        at com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)

        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:149)

        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:193)

        at $Proxy11.getFileInfo(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)

        at $Proxy11.getFileInfo(Unknown Source)

        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:629)

        at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1545)

        at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:820)

        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1378)

        at org.apache.hive.service.cli.CLIService.setupScratchDir(CLIService.java:458)

        at org.apache.hive.service.cli.CLIService.start(CLIService.java:87)

        ... 8 more

 

could you pls help me to have a check?

Re: hi i met issue when upgrading hive from 0.10 to 0.11.

There are inconsistencies present when trying to run Hive from CDH5 with
HDFS from CDH4 and HS2 might not start. This is not a supported combination
of software. Please consider upgrading the entire cluster or rolling back
to the earlier version of Hive.

Regards,
Gautam Gopalakrishnan