question HDP 3.1 Hive Connectivity Issue in Support Questions

question HDP 3.1 Hive Connectivity Issue in Support Questions https://community.cloudera.com/t5/Support-Questions/HDP-3-1-Hive-Connectivity-Issue/m-p/285115#M211621 I am migrating the spark jobs running in HDP 2.6 to HDP 3.1. When executing the spark jobs in HDP3.1 I am getting the following error.       <LI-CODE lang="markup">java.util.NoSuchElementException: spark.sql.hive.hiveserver2.jdbc.url at org.apache.spark.sql.internal.SQLConf$$anonfun$getConfString$2.apply(SQLConf.scala:1571) at org.apache.spark.sql.internal.SQLConf$$anonfun$getConfString$2.apply(SQLConf.scala:1571) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.sql.internal.SQLConf.getConfString(SQLConf.scala:1571) at org.apache.spark.sql.RuntimeConfig.get(RuntimeConfig.scala:74) at com.hortonworks.spark.sql.hive.llap.HWConf.getConnectionUrlFromConf(HWConf.java:143) at com.hortonworks.spark.sql.hive.llap.HWConf.getConnectionUrl(HWConf.java:107) at com.hortonworks.spark.sql.hive.llap.HiveWarehouseBuilder.build(HiveWarehouseBuilder.java:97) at com.wunderman.hdp.Hdp3MigrationMain.main(Hdp3MigrationMain.java:16) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:904) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)</LI-CODE>         The way I m creating the spark session        <LI-CODE lang="markup">private static SparkSession getSparkSession() { /** * Create an instance of SparkSession to connect to the cluster */ SparkSession sparkSession = SparkSession.builder().appName("Hdp3 Migration").master("yarn").getOrCreate(); return sparkSession; }</LI-CODE>        But the hiveserver2 jdbc url is configured in the spark config.  I have added the following dependency in the pom.xml       <LI-CODE lang="markup"><dependency> <groupId>com.hortonworks.hive</groupId> <artifactId>hive-warehouse-connector_2.11</artifactId> <version>1.0.0.3.1.0.0-78</version> </dependency></LI-CODE>     And I am trying to execute the below code     <LI-CODE lang="markup">String hdp3Enabled = args[0]; Dataset<Row> dataset; String query="SELECT * FROM schema.tablename where col1='abc' ; //Sample query try { if ("Y".equalsIgnoreCase(hdp3Enabled)) { HiveWarehouseSession hive = HiveWarehouseSession.session(sparkSession).build(); dataset = hive.executeQuery(query); } else { dataset = sparkSession.sql(query); } dataset.show(); } catch(Exception e) { e.printStackTrace(); }</LI-CODE>      Share your suggestions to fix the issue. Mon, 09 Dec 2019 13:49:20 GMT eswarloges 2019-12-09T13:49:20Z