Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH-5.4.2-1 - Spark Thrift Server Error

CDH-5.4.2-1 - Spark Thrift Server Error

New Contributor

Hello, 

 

I am trying to run thriftserver packaged within CDH-5.4.2-1, but it is throwing the following error.

Are there any ways or shortcuts to run Spark ThriftServer in Cloudera Package.

 

failed to launch org.apache.spark.sql.hive.thriftserver.HiveThriftServer2:
Failed to load main class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.
You need to build Spark with -Phive and -Phive-thriftserver.

 

Please help me out.

 

 

Thanks

 

Sathish

2 REPLIES 2

Re: CDH-5.4.2-1 - Spark Thrift Server Error

Master Guru
"""
Certain Spark SQL features not supported
The following Spark SQL features are not supported:
Thrift JDBC/ODBC server
Spark SQL (DataFrames) in PySpark
""" - http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cdh_rn_spark_ki.htm...

CDH does not ship or support Spark's thrift server, and recommends the use of Impala instead.

Like the error suggests, you'll need to manually build a CDH spark source tar-ball with the mentioned flags, to obtain it. The 5.4.2 sources for Spark are at http://archive.cloudera.com/cdh5/cdh/5/spark-1.3.0-cdh5.4.2-src.tar.gz

Re: CDH-5.4.2-1 - Spark Thrift Server Error

New Contributor

Harsh,

 

could you please elaborate how to integrate Spark Streaming and Impala to achieve the same functionality as HiveThriftServer2 provides? I would like to register a DF as a temp table and be able to access it via Impala, is there a way to do so without lauching Thrift Server2 that Cloudera does not support?

 

Thanks,

Anna