Even with HDP 3.1.5 that allow sharing same metastore for Spark and Hive, if you still want to use HWC to connect to Hive manage tables, you'll need LLAP available. This is required because Spark executors speaks directly with LLAP daemons using HWC to be more efficient. See documentations below that explains the mechanism: https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/developing-spark-applications/content/using_spark_hive_warehouse_connector.html https://blog.cloudera.com/hive-warehouse-connector-use-cases/ ... View more

Hello @sduraisankar93,   If you are facing this issue, as you said it's because you do not have imported the module.   I believe you should check this documentation on how to import HWC and use it : https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/integrating-hive/content/hive_configure_a_spark_hive_connection.html   If you are using Zeppelin, please check this : https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/integrating-hive/content/hive_zeppelin_configuration_hivewarehouseconnector.html   Note that, in Zeppelin pysark configuration could not work, so a work around is to set (via Ambari) in Zeppelin-env.sh section, this configuration : export SPARK_SUBMIT_OTPIONS="--jars usr/hdp/current/hive_warehouse_connector/hive-warehouse-connector-assembly-<version>.jar --py-files usr/hdp/current/hive_warehouse_connector/hive-warehouse-connector-assembly-<version>.zip"   Then to start a pyspark shell on your machines, launch this command : pyspark --jars /usr/hdp/current/hive_warehouse_connector/hive-warehouse-connector-assembly-jar --py-files /usr/hdp/current/hive_warehouse_connector/pyspark_hwc-.zip ... View more