Spark Connect to CDP Warehouse using Hive JDBC Method not supported error

[#1]

Hello, I've been trying to use Spark jdbc to connect to a CDP cloud virtual warehouse using the Hive jar (hive-jdbc-3.1.0-SNAPSHOT-standalone.jar) but I have the following error

Caused by: java.sql.SQLFeatureNotSupportedException: Method not supported
at org.apache.hive.jdbc.HivePreparedStatement.addBatch(HivePreparedStatement.java:78)

Looking into the jar sources I see that the addBatch method throws the exception by default. Spark needs to use this method to insert the data into hive. Is there any other driver that can be used to overcome this problem.If no, does cloudera will support spark to connect to CDP virtual warehouse.

Thanks in advance

Best regards,

Diego

[#2]

@dcardenas Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Spark experts @Bharati and @jagadeesan  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3 · Accepted solution]

Hi @dcardenas 
In CDH/CDP clusters, the integration between Spark and Hive service is get done by implementing Spark SQL API for Hive External tables through HiveMetastore and HWC (Hive Warehouse Connector) to Hive Managed/ACID tables.

Therefore, Spark accessing Hive with JDBC is not supported, please check Unsupported Apache Spark Features