Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark and hive problem in hdp 2.4

Spark and hive problem in hdp 2.4


Re: Spark and hive problem in hdp 2.4

Super Mentor

@juan pedro barbancho manchón

The Error indicates that you might not have added the hive-site.xml (like: /etc/spark/conf/hive-site.xml) and required jar in spark-submit command.

Exception: ("You must build Spark with Hive. Export 'SPARK_HIVE=true' and run build/sbt assembly", Py4JJavaError(u'An error occurred while calling\n', JavaObject id=o50))


Usually we specify the hive-site.xml via command line as :

# spark-submit --master yarn-cluster  --files /etc/spark/conf/hive-site.xml    .........


Don't have an account?
Coming from Hortonworks? Activate your account here