Hi, I am trying to submit a spark pi example in yarn-client mode and i am getting a weird error. HDP 2.4.2 and ambari 2.2.2 command : ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster --driver-memory 4g --executor-memory 2g --executor-cores 4 --queue default lib/spark-examples*.jar 100 Error: INFO Client: Using the spark assembly jar on HDFS because you are using HDP, defaultSparkAssembly:hdfs://clusername/hdp/apps/2.4.2.0-258/spark/spark-hdp-assembly.jar Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1 at org.apache.spark.deploy.yarn.YarnSparkHadoopUtil$anonfun$setEnvFromInputString$1.apply(YarnSparkHadoopUtil.scala:440) at org.apache.spark.deploy.yarn.YarnSparkHadoopUtil$anonfun$setEnvFromInputString$1.apply(YarnSparkHadoopUtil.scala:438) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108) at org.apache.spark.deploy.yarn.YarnSparkHadoopUtil$.setEnvFromInputString(YarnSparkHadoopUtil.scala:438) at org.apache.spark.deploy.yarn.Client$anonfun$setupLaunchEnv$6.apply(Client.scala:643) at org.apache.spark.deploy.yarn.Client$anonfun$setupLaunchEnv$6.apply(Client.scala:641) at scala.Option.foreach(Option.scala:236) at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:641) at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:732) at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:143) at org.apache.spark.deploy.yarn.Client.run(Client.scala:1079) at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1139) at org.apache.spark.deploy.yarn.Client.main(Client.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) I am also getting same error when i try to restart the spark thrift server from ambari. Please help me fix this issue ... View more

we are currently using Hive view version 1.0 and ambari 2.2.2.0. I have integrated with LDAP and whenever i click on hive view it asks for credentials and i can successfully login and query it. But my ldap password changes for every 4 hours and hive view does not refresh or ask for my ldap credentials again which result in 401 error. Can some one please help me how should i fix this issue. It works fine as long as the initial password is valid. I have tried to clear the browser cache but did not work. How exactly does hive view authenticate ... View more