Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oracle 12c support for hive metastore

Solved Go to solution
Highlighted

Oracle 12c support for hive metastore

New Contributor

Installed HDP2.4.2 with custom users. Hive metastore database used oracle 12c. metastore starting and failing with below error.

used both ojdbc6.jar and ojdbc7.jar.

2017-01-04 22:33:25,281 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:main(5917)) - Starting hive metastore on port 9083 2017-01-04 22:33:25,379 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(179)) - Starting JVM pause monitor 2017-01-04 22:33:25,404 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(599)) - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2017-01-04 22:33:25,434 INFO [main]: metastore.ObjectStore (ObjectStore.java:initialize(294)) - ObjectStore, initialize called 2017-01-04 22:33:26,699 INFO [main]: metastore.ObjectStore (ObjectStore.java:getPMF(379)) - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,Database,Type,FieldSchema,Order" 2017-01-04 22:33:46,499 INFO [main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(140)) - Using direct SQL, underlying DB is ORACLE 2017-01-04 22:33:46,502 INFO [main]: metastore.ObjectStore (ObjectStore.java:setConf(277)) - Initialized ObjectStore 2017-01-04 22:34:42,882 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(196)) - Detected pause in JVM or host machine (eg GC): pause of approximately 3368ms GC pool 'PS MarkSweep' had collection(s): count=1 time=3444ms GC pool 'PS Scavenge' had collection(s): count=1 time=353ms 2017-01-04 22:36:15,411 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(196)) - Detected pause in JVM or host machine (eg GC): pause of approximately 2512ms GC pool 'PS MarkSweep' had collection(s): count=1 time=1756ms GC pool 'PS Scavenge' had collection(s): count=1 time=1136ms 2017-01-04 22:37:11,339 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(196)) - Detected pause in JVM or host machine (eg GC): pause of approximately 2400ms GC pool 'PS MarkSweep' had collection(s): count=1 time=2518ms 2017-01-04 22:41:02,389 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(196)) - Detected pause in JVM or host machine (eg GC): pause of approximately 5415ms GC pool 'PS MarkSweep' had collection(s): count=1 time=5109ms GC pool 'PS Scavenge' had collection(s): count=2 time=626ms 2017-01-04 22:41:27,052 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(196)) - Detected pause in JVM or host machine (eg GC): pause of approximately 5658ms GC pool 'PS MarkSweep' had collection(s): count=1 time=5718ms 2017-01-04 22:44:52,288 WARN [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(192)) - Detected pause in JVM or host machine (eg GC): pause of approximately 16503ms GC pool 'PS MarkSweep' had collection(s): count=2 time=15796ms GC pool 'PS Scavenge' had collection(s): count=3 time=959ms 2017-01-04 22:44:52,291 WARN [main]: metastore.HiveMetaStore (HiveMetaStore.java:createDefaultDB(632)) - Retrying creating default database after error: Exception thrown while loading remaining rows of query javax.jdo.JDOUserException: Exception thrown while loading remaining rows of query at org.datanucleus.api.jdo.JDOAdapter.getUserExceptionForException(JDOAdapter.java:1139) at org.datanucleus.store.rdbms.query.ForwardQueryResult.closingConnection(ForwardQueryResult.java:296) at org.datanucleus.store.query.AbstractQueryResult.disconnect(AbstractQueryResult.java:107) at org.datanucleus.store.rdbms.query.AbstractRDBMSQueryResult.disconnect(AbstractRDBMSQueryResult.java:236) at org.datanucleus.store.rdbms.query.JDOQLQuery$2.transactionPreClose(JDOQLQuery.java:755) at org.datanucleus.store.connection.AbstractManagedConnection.transactionPreClose(AbstractManagedConnection.java:94) at org.datanucleus.store.connection.ConnectionManagerImpl$2.transactionPreRollBack(ConnectionManagerImpl.java:447) at org.datanucleus.TransactionImpl.internalPreRollback(TransactionImpl.java:500) at org.datanucleus.TransactionImpl.rollback(TransactionImpl.java:427) at org.datanucleus.api.jdo.JDOTransaction.rollback(JDOTransaction.java:182) at org.apache.hadoop.hive.metastore.ObjectStore.rollbackTransaction(ObjectStore.java:505) at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6763) at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6732) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6690) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6674) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114) at com.sun.proxy.$Proxy2.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:582) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:630) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:470) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5792) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5787) at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6020) at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:5945) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) NestedThrowablesStackTrace: java.lang.NullPointerException 2017-01-04 22:44:52,292 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(599)) - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2017-01-04 22:44:52,293 INFO [main]: metastore.ObjectStore (ObjectStore.java:initialize(294)) - ObjectStore, initialize called 2017-01-04 22:44:52,303 INFO [main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(140)) - Using direct SQL, underlying DB is ORACLE 2017-01-04 22:44:52,304 INFO [main]: metastore.ObjectStore (ObjectStore.java:setConf(277)) - Initialized ObjectStore 2017-01-04 22:46:30,360 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(196)) - Detected pause in JVM or host machine (eg GC): pause of approximately 1676ms GC pool 'PS MarkSweep' had collection(s): count=1 time=1272ms GC pool 'PS Scavenge' had collection(s): count=1 time=530ms 2017-01-04 22:48:07,328 INFO [org.apache.hadoop.util.JvmPauseMonitor$Monitor@74011c74]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(196)) - Detected pause in JVM or host machine (eg GC): pause of approximately 3741ms GC pool 'PS MarkSweep' had collection(s): count=1 time=2501ms GC pool 'PS Scavenge' had collection(s): count=1 time=1283ms 2017-01-04 22:51:54,842 INFO org.apache.hadoop.util.RunJar.main(RunJar.java:136) NestedThrowablesStackTrace: java.lang.NullPointerException 2017-01-04 23:44:47,777 INFO [Thread-4]: metastore.HiveMetaStore (HiveMetaStore.java:run(5934)) - Shutting down hive metastore.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Oracle 12c support for hive metastore

New Contributor

Fixed the issue by downloading exact version of ojbc for the oracle version(11.2.0.4). We should not use any other ojdbc version.

View solution in original post

3 REPLIES 3
Highlighted

Re: Oracle 12c support for hive metastore

Super Guru

Dup post here

https://community.hortonworks.com/questions/29320/critical-alerts-for-hive-metastore-hive-not-able-t...

Seems not able to get it working with 12c (without HWX support assistance). the post above used mysql instead.

Highlighted

Re: Oracle 12c support for hive metastore

New Contributor

Client dont want to use other than oracle. Is there any patch released?

Highlighted

Re: Oracle 12c support for hive metastore

New Contributor

Fixed the issue by downloading exact version of ojbc for the oracle version(11.2.0.4). We should not use any other ojdbc version.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here