<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>question getting exception in spark-hive application. in Support Questions</title>
    <link>https://community.cloudera.com/t5/Support-Questions/getting-exception-in-spark-hive-application/m-p/168520#M130850</link>
    <description>&lt;P&gt;The exception stack trace is as:&lt;/P&gt;&lt;P&gt;6/11/04 10:08:46 INFO YarnAllocator: Will request 2 executor containers, each with 1 cores and 1408 MB memory including 384 MB overhead
16/11/04 10:08:46 INFO YarnAllocator: Container request (host: Any, capability: &amp;lt;memory:1408, vCores:1&amp;gt;)
16/11/04 10:08:46 INFO YarnAllocator: Container request (host: Any, capability: &amp;lt;memory:1408, vCores:1&amp;gt;)
16/11/04 10:08:46 INFO ApplicationMaster: Started progress reporter thread with (heartbeat : 5000, initial allocation : 200) intervals
16/11/04 10:09:15 INFO YarnClusterSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000(ms)
16/11/04 10:09:15 INFO YarnClusterScheduler: YarnClusterScheduler.postStartHook done
16/11/04 10:09:16 INFO HiveContext: Initializing execution hive, version 1.2.1
16/11/04 10:09:16 INFO ClientWrapper: Inspected Hadoop version: 2.7.3.2.5.0.0-1245
16/11/04 10:09:16 INFO ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.7.3.2.5.0.0-1245
16/11/04 10:09:17 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/11/04 10:09:17 INFO ObjectStore: ObjectStore, initialize called
16/11/04 10:09:17 INFO Persistence: Property datanucleus.fixedDatastore unknown - will be ignored
16/11/04 10:09:17 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
16/11/04 10:09:17 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored
16/11/04 10:09:22 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
16/11/04 10:09:23 INFO JDO: Exception thrown
Identifier name is unresolved (not a static field)
org.datanucleus.exceptions.NucleusUserException: Identifier name is unresolved (not a static field)
   at org.datanucleus.query.expression.PrimaryExpression.bind(PrimaryExpression.java:182)
   at org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:89)
   at org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:492)
   at org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:112)
   at org.datanucleus.store.query.AbstractJDOQLQuery.compileGeneric(AbstractJDOQLQuery.java:366)
   at org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:411)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:236)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   ...
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN MetaStoreDirectSql: Database initialization failed; direct SQL is disabled
javax.jdo.JDOUserException: Identifier name is unresolved (not a static field)
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:636)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:245)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
   at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
   at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.&amp;lt;init&amp;gt;(RawStoreProxy.java:57)
....
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
NestedThrowablesStackTrace:
Identifier name is unresolved (not a static field)
org.datanucleus.exceptions.NucleusUserException: Identifier name is unresolved (not a static field)
   at org.datanucleus.query.expression.PrimaryExpression.bind(PrimaryExpression.java:182)
   at org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:89)
   at org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:492)
   at org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:112)
   at org.datanucleus.store.query.AbstractJDOQLQuery.compileGeneric(AbstractJDOQLQuery.java:366)
   at org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:411)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:236)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   ...
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 INFO ObjectStore: Initialized ObjectStore
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   ....
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   ...
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN HiveMetaStore: Retrying creating default database after error: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:473)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:735)
   at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:755)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6773)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   ..
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
NestedThrowablesStackTrace:
The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
org.datanucleus.exceptions.ClassNotPersistableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5506)
   at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:1966)
   at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1909)
   at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1764)
   at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:730)
   at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:755)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6773)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
...
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/11/04 10:09:23 INFO ObjectStore: ObjectStore, initialize called
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MDatabase and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MDatabase" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MDatabase" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
....
   at org.apache.hadoop.hive.ql.metadata.Hive.&amp;lt;clinit&amp;gt;(Hive.java:166)
   at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
   at org.apache.spark.sql.hive.client.ClientWrapper.&amp;lt;init&amp;gt;(ClientWrapper.scala:204)
   at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MTableColumnStatistics and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MTableColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MTableColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:184)
   .....
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
...
   at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
   at org.apache.spark.sql.hive.client.ClientWrapper.&amp;lt;init&amp;gt;(ClientWrapper.scala:204)
   at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN MetaStoreDirectSql: Self-test query [select "DB_ID" from "DBS"] failed; direct SQL is disabled
javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS"".
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:245)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.runTestQuery(MetaStoreDirectSql.java:199)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
   at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
   at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.&amp;lt;init&amp;gt;(RawStoreProxy.java:57)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
   ....
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
NestedThrowablesStackTrace:
java.sql.SQLSyntaxErrorException: Table/View 'DBS' does not exist.
   at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.&amp;lt;init&amp;gt;(Unknown Source)
...
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
Caused by: java.sql.SQLException: Table/View 'DBS' does not exist.
   at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
   ... 69 more
Caused by: ERROR 42X05: Table/View 'DBS' does not exist.
   at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source)
   at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source)
   at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source)
   at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)
   at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source)
   at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
   at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
   at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
   ... 63 more
16/11/04 10:09:23 INFO ObjectStore: Initialized ObjectStore
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721)
   at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703)
....
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6763)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   ...
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN Hive: Failed to access metastore. This class should not accessed in runtime.
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
   at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
   at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
   at org.apache.hadoop.hive.ql.metadata.Hive.&amp;lt;clinit&amp;gt;(Hive.java:166)
   at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
   at org.apache.spark.sql.hive.client.ClientWrapper.&amp;lt;init&amp;gt;(ClientWrapper.scala:204)
   at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
   at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
   at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.&amp;lt;init&amp;gt;(RetryingMetaStoreClient.java:86)
   at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
   at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
   at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
   at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
   at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
   ... 19 more
Caused by: java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
   ... 25 more
Caused by: org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
NestedThrowables:
org.datanucleus.exceptions.ClassNotPersistableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:473)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:735)
   at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:755)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6773)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
   at com.sun.proxy.$Proxy25.verifySchema(Unknown Source)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:572)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
   at org.apache.hadoop.hive.metastore.RetryingHMSHandler.&amp;lt;init&amp;gt;(RetryingHMSHandler.java:66)
   at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
   at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
   at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&amp;lt;init&amp;gt;(HiveMetaStoreClient.java:199)
   at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.&amp;lt;init&amp;gt;(SessionHiveMetaStoreClient.java:74)
   ... 30 more
Caused by: org.datanucleus.exceptions.ClassNotPersistableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5506)
   at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:1966)
   at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1909)
   at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1764)
   at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:730)
   ... 48 more
16/11/04 10:09:23 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/11/04 10:09:23 INFO ObjectStore: ObjectStore, initialize called
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MDatabase and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MDatabase" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MDatabase" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   ...
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MTableColumnStatistics and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MTableColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MTableColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:184)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
...
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:185)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   ...
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN MetaStoreDirectSql: Self-test query [select "DB_ID" from "DBS"] failed; direct SQL is disabled
javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS"".
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:245)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.runTestQuery(MetaStoreDirectSql.java:199)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
   at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
   at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.&amp;lt;init&amp;gt;(RawStoreProxy.java:57)
   ...
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
NestedThrowablesStackTrace:
java.sql.SQLSyntaxErrorException: Table/View 'DBS' does not exist.
   at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown Source)
...
   at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
Caused by: java.sql.SQLException: Table/View 'DBS' does not exist.
   at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
   ... 66 more
Caused by: ERROR 42X05: Table/View 'DBS' does not exist.
   at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source)
   at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source)
   at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source)
   at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)
   at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source)
   at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
   at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
   at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
   ... 60 more
16/11/04 10:09:23 INFO ObjectStore: Initialized ObjectStore
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   ...
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6763)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   ....
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN HiveMetaStore: Retrying creating default database after error: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:473)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:735)
   at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:755)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6773)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   ....
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
NestedThrowablesStackTrace:
The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
org.datanucleus.exceptions.ClassNotPersistableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5506)
   at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:1966)
   at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1909)
   at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1764)
   at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:730)
   at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:755)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6773)
...
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/11/04 10:09:23 INFO ObjectStore: ObjectStore, initialize called
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MDatabase and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MDatabase" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MDatabase" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
   at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
   at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
....
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MTableColumnStatistics and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MTableColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MTableColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:184)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
   at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
..
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:185)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   ...
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN MetaStoreDirectSql: Self-test query [select "DB_ID" from "DBS"] failed; direct SQL is disabled
javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS"".
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:245)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.runTestQuery(MetaStoreDirectSql.java:199)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
   at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
   at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.&amp;lt;init&amp;gt;(RawStoreProxy.java:57)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
...
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
NestedThrowablesStackTrace:
java.sql.SQLSyntaxErrorException: Table/View 'DBS' does not exist.
   at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.&amp;lt;init&amp;gt;(Unknown Source)
   at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
   at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
   at com.jolbox.bonecp.ConnectionHandle.prepareStatement(ConnectionHandle.java:1193)
   at org.datanucleus.store.rdbms.SQLController.getStatementForQuery(SQLController.java:345)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getPreparedStatementForQuery(RDBMSQueryUtils.java:211)
   at org.datanucleus.store.rdbms.query.SQLQuery.performExecute(SQLQuery.java:633)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1840)
   at org.datanucleus.store.rdbms.query.SQLQuery.executeWithArray(SQLQuery.java:807)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.runTestQuery(MetaStoreDirectSql.java:199)
   at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.&amp;lt;init&amp;gt;(MetaStoreDirectSql.java:137)
   at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
   at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
   at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
   at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
   at org.apache.hadoop.hive.metastore.RawStoreProxy.&amp;lt;init&amp;gt;(RawStoreProxy.java:57)
...
   at org.apache.spark.sql.hive.client.ClientWrapper.&amp;lt;init&amp;gt;(ClientWrapper.scala:204)
   at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
Caused by: java.sql.SQLException: Table/View 'DBS' does not exist.
   at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
   at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
   ... 66 more
Caused by: ERROR 42X05: Table/View 'DBS' does not exist.
   at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source)
   at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source)
   at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source)
   at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source)
   at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)
   at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source)
   at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
   at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
   at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
   ... 60 more
16/11/04 10:09:23 INFO ObjectStore: Initialized ObjectStore
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721)
   at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6661)
   at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   ...
   at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
   at org.apache.spark.sql.hive.client.ClientWrapper.&amp;lt;init&amp;gt;(ClientWrapper.scala:204)
   at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
16/11/04 10:09:23 WARN Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates
Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
org.datanucleus.store.rdbms.exceptions.NoTableManagedException: Persistent class "org.apache.hadoop.hive.metastore.model.MVersionTable" has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.
   at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:694)
   at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:896)
   at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:344)
   at org.datanucleus.store.query.Query.executeQuery(Query.java:1801)
   at org.datanucleus.store.query.Query.executeWithArray(Query.java:1729)
   at org.datanucleus.store.query.Query.execute(Query.java:1711)
   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:236)
   at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6763)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   ...
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
16/11/04 10:09:23 ERROR ApplicationMaster: User class threw exception: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
   at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
   at org.apache.spark.sql.hive.client.ClientWrapper.&amp;lt;init&amp;gt;(ClientWrapper.scala:204)
   at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:225)
   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:215)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:480)
   at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:479)
   at org.apache.spark.sql.UDFRegistration.&amp;lt;init&amp;gt;(UDFRegistration.scala:40)
   at org.apache.spark.sql.SQLContext.&amp;lt;init&amp;gt;(SQLContext.scala:330)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:90)
   at org.apache.spark.sql.hive.HiveContext.&amp;lt;init&amp;gt;(HiveContext.scala:101)
   at io.woolford.Main.main(Main.java:21)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:559)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
   at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
   at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.&amp;lt;init&amp;gt;(RetryingMetaStoreClient.java:86)
   at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
   at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
   at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
   at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
   at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
   ... 16 more
Caused by: java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
   ... 22 more
Caused by: org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
NestedThrowables:
org.datanucleus.exceptions.ClassNotPersistableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:473)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:735)
   at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:755)
   at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6773)
   at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670)
   at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
...)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
   at org.apache.hadoop.hive.metastore.RetryingHMSHandler.&amp;lt;init&amp;gt;(RetryingHMSHandler.java:66)
   at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
   at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
   at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&amp;lt;init&amp;gt;(HiveMetaStoreClient.java:199)
   at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.&amp;lt;init&amp;gt;(SessionHiveMetaStoreClient.java:74)
   ... 27 more
Caused by: org.datanucleus.exceptions.ClassNotPersistableException: The class "org.apache.hadoop.hive.metastore.model.MVersionTable" is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found.
   at org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5506)
   at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:1966)
   at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1909)
   at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1764)
   at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:730)
   ... 45 more
16/11/04 10:09:23 INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient)
16/11/04 10:09:23 INFO SparkContext: Invoking stop() from shutdown hook
16/11/04 10:09:23 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/static/sql,null}
16/11/04 10:09:23 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution/json,null}
16/11/04 10:09:23 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution,null}&lt;/P&gt;</description>
    <pubDate>Tue, 08 Nov 2016 16:54:13 GMT</pubDate>
    <dc:creator>er_sanjay_2011</dc:creator>
    <dc:date>2016-11-08T16:54:13Z</dc:date>
  </channel>
</rss>

