Skip to main content
sinhapiyush86
sinhapiyush86
Bronze ⭐️⭐️⭐️
March 2, 2020
Solved

Spark-Hive connection issue (Unsupported JDBC protocol: 'null')

py4j.protocol.Py4JJavaError: An error occurred while calling o66.save.
: java.lang.RuntimeException: java.lang.IllegalArgumentException: Unsupported JDBC protocol: 'null'
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseDataSourceWriter.createInternalRowWriterFactory(HiveWarehouseDataSourceWriter.java:102)
at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec.doExecute(WriteToDataSourceV2.scala:55)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:656)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:656)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:656)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:256)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Unsupported JDBC protocol: 'null'
at com.hortonworks.spark.sql.hive.llap.JDBCWrapper$$anonfun$getDriverClass$2.apply(HS2JDBCWrapper.scala:89)
at com.hortonworks.spark.sql.hive.llap.JDBCWrapper$$anonfun$getDriverClass$2.apply(HS2JDBCWrapper.scala:75)
at scala.Option.getOrElse(Option.scala:121)
at com.hortonworks.spark.sql.hive.llap.JDBCWrapper.getDriverClass(HS2JDBCWrapper.scala:74)
at com.hortonworks.spark.sql.hive.llap.JDBCWrapper.getConnector(HS2JDBCWrapper.scala:307)
at com.hortonworks.spark.sql.hive.llap.DefaultJDBCWrapper.getConnector(HS2JDBCWrapper.scala)
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseDataSourceWriter.getConnection(HiveWarehouseDataSourceWriter.java:176)
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseDataSourceWriter.createInternalRowWriterFactory(HiveWarehouseDataSourceWriter.java:95)
... 25 more

 

 

 

My Pyspark statement creating error.

df.write.format("com.hortonworks.spark.sql.hive.llap.HiveWarehouseConnector").option("table","table_name").save()
 
I have properly supplied mysql-java.jar, hive-warehouse-connector-assembly-1.0.0.3.1.0.0-78.jar ,pyspark_hwc-1.0.0.3.1.0.0-78.zip
 
Can someone look into it?
Bender
BenderAnswer
Gold ⭐️⭐️⭐️
April 21, 2020

Hello @sinhapiyush86 ,

 

thank you for raising the question about getting Unsupported JDBC protocol: 'null' exception in PySpark.

 

Please make sure you have initialised HWC in the session, otherwise you will get the below exception:
java.lang.RuntimeException: java.lang.IllegalArgumentException: Unsupported JDBC protocol: 'null'

 

You can initialise HWC by the below code segment [1]:
from pyspark_llap import HiveWarehouseSession

hive = HiveWarehouseSession.session(spark).build()

 

Please let us know if it resolved your issue.

 

Thank you:

Ferenc

[1] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/integrating-hive/content/hive_hivewarehousesession_api_operations.html

Ferenc Erdelyi, Technical Solutions Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. Learn more about the Cloudera Community: - Community Guidelines (https://community.cloudera.com/t5/custom/page/page-id/Community_Guidelines) - How to use the forum (/t5/help/faqpage)
View in thread

1 reply

Bender
Gold ⭐️⭐️⭐️
BenderAnswer
Gold ⭐️⭐️⭐️
April 21, 2020

Hello @sinhapiyush86 ,

 

thank you for raising the question about getting Unsupported JDBC protocol: 'null' exception in PySpark.

 

Please make sure you have initialised HWC in the session, otherwise you will get the below exception:
java.lang.RuntimeException: java.lang.IllegalArgumentException: Unsupported JDBC protocol: 'null'

 

You can initialise HWC by the below code segment [1]:
from pyspark_llap import HiveWarehouseSession

hive = HiveWarehouseSession.session(spark).build()

 

Please let us know if it resolved your issue.

 

Thank you:

Ferenc

[1] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/integrating-hive/content/hive_hivewarehousesession_api_operations.html

Translate
Ferenc Erdelyi, Technical Solutions Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. Learn more about the Cloudera Community: - Community Guidelines (https://community.cloudera.com/t5/custom/page/page-id/Community_Guidelines) - How to use the forum (/t5/help/faqpage)
    Terms & ConditionsAccessibility statement