While running the spark-bench command ./examples/multi-submit-sparkpi/multi-submit-example.sh from spark bench distribution file, getting this error
Exception in thread "main" java.lang.Exception: spark-submit failed to complete properly given these arguments:
--class com.ibm.sparktc.sparkbench.cli.CLIKickoff --master yarn-client /home/dialdev/spark-bench_2.1.1_0.2.2-RELEASE/lib/spark-bench-2.1.1_0.2.2-RELEASE.jar /tmp/spark-bench-3184271760086441425.conf
at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$.launch(SparkLaunch.scala:47)
at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$anonfun$run$2.apply(SparkLaunch.scala:39)
at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$anonfun$run$2.apply(SparkLaunch.scala:39)
at scala.collection.immutable.List.foreach(List.scala:381)
at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$.run(SparkLaunch.scala:39)
at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$.main(SparkLaunch.scala:16)
at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch.main(SparkLaunch.scala)
In spark-bench-env.sh, the environment variables are set as follows:
export SPARK_HOME=/usr/hdp/2.5.3.0-37/spark
export SPARK_MASTER_HOST=yarn-client (also tried with yarn)
Thanks in advance