Reply
Highlighted
Explorer
Posts: 17
Registered: ‎06-03-2017

hive context on spark in quickstart vm

I am unable to initialize hive context in spark. Hive runs as standalone

 

I changed the hive-site.xml in spark/conf as:

  <property>    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>    Local or HDFS directory where Hive keeps table contents.
    </description>  </property>
  <property>   <name>javax.jdo.PersistenceManagerFactoryClass</name>
   <value>org.datanucleus.api.jdo.JDOPersistenceManagerFactory</value>
   <description>class implementing the jdo persistence</description>  </property>

  <!--<property>   <name>hive.enable.spark.execution.engine</name>
   <value>true</value>  </property> -->

  <!-- modded   -->
  <property>    <name>javax.jdo.option.ConnectionURL</name>
    <!--<value>jdbc:mysql://127.0.0.1/metastore?createDatabaseIfNotExist=true</value>-->
   <value>jdbc:derby:;databaseName=/home/cloudera/metastore_db;create=true</value>
    <description>JDBC connect string for a JDBC metastore</description>  </property>

    <!--  modded   -->
  <property>    <name>javax.jdo.option.ConnectionDriverName</name>
    <!--<value>com.mysql.jdbc.Driver</value>-->
   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
    <description>Driver class name for a JDBC metastore</description>  </property>

 <!-- <property>    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>  </property>
  <property>    <name>javax.jdo.option.ConnectionPassword</name>
    <value>cloudera</value>  </property>  -->

<property> <name>hive.hwi.war.file</name> <value>/usr/lib/hive/lib/hive-hwi-0.8.1-cdh4.0.0.jar</value> <description>This is the WAR file with the jsp content for Hive Web Interface</description> </property> <property> <name>datanucleus.fixedDatastore</name> <value>true</value> </property> <property> <name>datanucleus.autoCreateSchema</name> <value>false</value> </property> <property> <name>hive.metastore.schema.verification</name> <value>true</value> </property> <property> <name>hive.metastore.uris</name> <value>thrift://127.0.0.1:9083</value> <description>IP address (or fully-qualified domain name) and port of the metastore host</description> </property> <!-- added by Deepak --> <property> <name>dfs.client.read.shortcircuit</name> <value>true</value> </property> </configuration>

In spark-shell:

import org.apache.spark.sql.hive.HiveContext
val hCtx= new org.apache.spark.sql.hive.HiveContext(sc)
val rCH= hCtx.hql("select * from mytable")

Errors:

17/08/24 17:29:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Spark context available as sc (master = local[*], app id = local-1503620964053).
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission denied)
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.replication.master in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.resultSetStatisticsFactory in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.NoneAuthentication in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.lockManagerJ6 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.log.readonly in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.jdbcJ8 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.resourceAdapterJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.validation in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.jdbcJ6 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.lcf in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.optimizer in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.XPLAINFactory in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.javaCompiler in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.tcf in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.dataDictionary in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.cacheManagerJ6 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.ef in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.dvfJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.dvfJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.daemon in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.cacheManagerJ6 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.data.genericJ4 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.timerJ1 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.rawStore.data.genericJ4 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.data.genericJ1 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.basicAuthentication in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.rawStore.data.genericJ1 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.classes.JNDIAuthentication in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.database in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.transaction in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.streams in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.classes.dvfJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.mgmt.jmx in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.nativeAuthentication in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.cryptographyJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.access.sort in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.classes.jdbcJ8 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.classes.cryptographyJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.classes.jdbcJ6 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.classes.rawStore.data.genericJ4 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.jdbcJ8 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.lockManagerJ6 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.cryptographyJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.jdbcJ6 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.specificAuthentication in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.mgmt.jmx in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.netServer.autoStart in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.access.uniquewithduplicatenullssort in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.classManagerJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.JNDIAuthentication in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.log in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.access.heap in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.database.slave in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.jdk.resourceAdapterJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.mgmt.null in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.env.classes.resourceAdapterJ2 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.access in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.replication.slave in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.lf in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.uuidJ1 in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.access.btree in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] Ignored duplicate property derby.module.rawStore in jar:file:/usr/lib/flume-ng/lib/derby-10.11.1.1.jar!/org/apache/derby/modules.properties
Thu Aug 24 17:29:28 PDT 2017 Thread[main,5,main] 
----------------------------------------------------------------
Thu Aug 24 17:29:28 PDT 2017:
Booting Derby version The Apache Software Foundation - Apache Derby - 10.11.1.1 - (1616546): instance a816c00e-015e-16cb-2ebd-00001f7c1fd0 
on database directory /tmp/spark-e1679bcc-90a7-4a58-b498-1b93233e5694/metastore with class loader sun.misc.Launcher$AppClassLoader@7d487b8b 
Loaded from file:/usr/lib/hive/lib/derby-10.11.1.1.jar
java.vendor=Oracle Corporation
java.runtime.version=1.7.0_67-b01
user.dir=/
os.name=Linux
os.arch=amd64
os.version=2.6.32-573.el6.x86_64
derby.system.home=null
Database Class Loader started - derby.database.classpath=''
17/08/24 17:29:37 WARN metadata.Hive: Failed to register all functions.
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1530)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:67)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)
	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3230)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3249)
	at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3474)
	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:225)
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:209)
	at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:332)
	at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:293)
	at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:268)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:529)
	at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:204)
	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
	at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:220)
	at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:210)
	at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:465)
	at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:464)
	at org.apache.spark.sql.UDFRegistration.<init>(UDFRegistration.scala:40)
	at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:342)
	at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:90)
	at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:101)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:1033)
	at $line4.$read$$iwC$$iwC.<init>(<console>:15)
	at $line4.$read$$iwC.<init>(<console>:24)
	at $line4.$read.<init>(<console>:26)
	at $line4.$read$.<init>(<console>:30)
	at $line4.$read$.<clinit>(<console>)
	at $line4.$eval$.<init>(<console>:7)
	at $line4.$eval$.<clinit>(<console>)
	at $line4.$eval.$print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1045)
	at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1326)
	at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:821)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:852)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:800)
	at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
	at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
	at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
	at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:133)
	at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)
	at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:305)
	at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)
	at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)
	at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:160)
	at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)
	at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)
	at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
	at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
	at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
	at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1064)
	at org.apache.spark.repl.Main$.main(Main.scala:35)
	at org.apache.spark.repl.Main.main(Main.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:730)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1528)
	... 73 more
Caused by: MetaException(message:Version information not found in metastore. )
	at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7160)
	at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7141)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
	at com.sun.proxy.$Proxy22.verifySchema(Unknown Source)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:627)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:675)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:484)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5995)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:203)
	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
	... 78 more
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:556)
	at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:204)
	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
	at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:220)
	at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:210)
	at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:465)
	at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:464)
	at org.apache.spark.sql.UDFRegistration.<init>(UDFRegistration.scala:40)
	at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:342)
	at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:90)
	at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:101)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:1033)
	at $iwC$$iwC.<init>(<console>:15)
	at $iwC.<init>(<console>:24)
	at <init>(<console>:26)
	at .<init>(<console>:30)
	at .<clinit>(<console>)
	at .<init>(<console>:7)
	at .<clinit>(<console>)
	at $print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1045)
	at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1326)
	at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:821)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:852)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:800)
	at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
	at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
	at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
	at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:133)
	at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)
	at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:305)
	at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)
	at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)
	at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:160)
	at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)
	at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)
	at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
	at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
	at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
	at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1064)
	at org.apache.spark.repl.Main$.main(Main.scala:35)
	at org.apache.spark.repl.Main.main(Main.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:730)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:214)
	at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:332)
	at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:293)
	at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:268)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:529)
	... 62 more
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1530)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:67)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)
	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3230)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3249)
	at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3474)
	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:225)
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:209)
	... 66 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1528)
	... 73 more
Caused by: MetaException(message:Version information not found in metastore. )
	at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7160)
	at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7141)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
	at com.sun.proxy.$Proxy22.verifySchema(Unknown Source)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:627)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:675)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:484)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5995)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:203)
	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
	... 78 more
<console>:16: error: not found: value sqlContext
         import sqlContext.implicits._
                ^
<console>:16: error: not found: value sqlContext
         import sqlContext.sql
                ^
scala> 

Can somebody help how to start hive context and load hive tables?

 

Appreciate your help in advance.