Hello, I did as directed above val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) sqlContext.setConf("yarn.timeline-service.enabled","false") now I have ended up with a new issue which is very similar to earlier one, but now its trying to cast from yarn client to yarn client,I dont see spark's assembly jar in error . let me know if any configuration changes are required . I am trying to run in cluster mode hdp 2.4.2.0-258 diagnostics: User class threw exception: java.lang.LinkageError:
ClassCastException: attempting to
castjar:file:/hadoop/hadoop/yarn/local/filecache/10020/spark-hdp-assembly.jar!/javax/ws/rs/ext/RuntimeDelegate.classtojar:file:/hadoop/hadoop/yarn/local/filecache/10020/spark-hdp-assembly.jar!/javax/ws/rs/ext/RuntimeDelegate.class Exception in thread "main" org.apache.spark.SparkException: Application
application_1499441914050_13463 finished with failed status
at
org.apache.spark.deploy.yarn.Client.run(Client.scala:1092)
at
org.apache.spark.deploy.yarn.Client$.main(Client.scala:1139)
at
org.apache.spark.deploy.yarn.Client.main(Client.scala)
at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$runMain(SparkSubmit.scala:731)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
17/07/13
12:38:12 INFO ShutdownHookManager: Shutdown hook
called
... View more
