Member since
11-26-2014
4
Posts
0
Kudos Received
0
Solutions
11-28-2014
08:18 AM
Thanks harsha_v, but I don't know how to add the hive client jars in my classpath, can u explain clear more? thank you
... View more
11-26-2014
09:34 PM
Hi, I installed Spark 1.1.0 and Hive 0.13, I try to run example code # sc is an existing SparkContext.
from pyspark.sql import HiveContext
sqlContext = HiveContext(sc)
sqlContext.sql("CREATE TABLE IF NOT EXISTS src (key INT, value STRING)")
sqlContext.sql("LOAD DATA LOCAL INPATH 'examples/src/main/resources/kv1.txt' INTO TABLE src")
# Queries can be expressed in HiveQL.
results = sqlContext.sql("FROM src SELECT key, value").collect() so I get error: Exception in thread "Thread-2" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at py4j.reflection.TypeUtil.getClass(TypeUtil.java:265) at py4j.reflection.TypeUtil.forName(TypeUtil.java:245) at py4j.commands.ReflectionCommand.getUnknownMember(ReflectionCommand.java:153) at py4j.commands.ReflectionCommand.execute(ReflectionCommand.java:82) at py4j.GatewayConnection.run(GatewayConnection.java:207) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 8 more Everyone can help me?
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
-
Apache Spark
-
Security