SPARK + LLAP not working in HDP 3.0?

subhash_parise3 — Mon, 24 Sep 2018 18:42:22 GMT

Hello Experts,

I am facing the below issue while loading hive from spark

scala> import com.hortonworks.hwc.HiveWarehouseSession

import com.hortonworks.hwc.HiveWarehouseSession

scala> import com.hortonworks.hwc.HiveWarehouseSession._

import com.hortonworks.hwc.HiveWarehouseSession._

scala> val hive1 = HiveWarehouseSession.session(spark).build()

java.util.NoSuchElementException: spark.sql.hive.hiveserver2.jdbc.url at org.apache.spark.sql.internal.SQLConf$anonfun$getConfString$2.apply(SQLConf.scala:1571) at org.apache.spark.sql.internal.SQLConf$anonfun$getConfString$2.apply(SQLConf.scala:1571) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.sql.internal.SQLConf.getConfString(SQLConf.scala:1571) at org.apache.spark.sql.RuntimeConfig.get(RuntimeConfig.scala:74) at com.hortonworks.spark.sql.hive.llap.HWConf.getConnectionUrlFromConf(HWConf.java:124) at com.hortonworks.spark.sql.hive.llap.HWConf.getConnectionUrl(HWConf.java:103) at com.hortonworks.spark.sql.hive.llap.HiveWarehouseBuilder.build(HiveWarehouseBuilder.java:97) ... 51 elided

I have configured spark llap by using the following url :

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/integrating-hive/content/hive_configure_a_spark_hive_connection.html

Any thing i am missing in this setup ? Any help much appreciated?

Re: SPARK + LLAP not working in HDP 3.0?

falbani — Mon, 24 Sep 2018 19:21:45 GMT

@subhash parise

Have you tried passing the configuration using --conf spark.sql.hive.hiveserver2.jdbc.url="jdbc url taken from ambari hive conf" ?

HTH

Re: SPARK + LLAP not working in HDP 3.0?

subhash_parise3 — Mon, 24 Sep 2018 20:01:17 GMT

Hi @Felix Albani,

I have configured required properties in custom spark-default config but still spark in not utilizing those properties.

i have used above syntax got worked for me.

question Re: SPARK + LLAP not working in HDP 3.0? in Archives of Support Questions (Read Only)

SPARK + LLAP not working in HDP 3.0?

Re: SPARK + LLAP not working in HDP 3.0?

Re: SPARK + LLAP not working in HDP 3.0?