Hi, I am trying to run a simple shell script that has the following content: spark-submit –class org.apache.spark.examples.SparkPi –master yarn-client /cloudera/opt/cloudera/parcels/CDH-5.4.7-1.cdh5.4.7.p0.3/lib/spark/lib/spark-examples.jar 100 In the shell command I write script.sh and then I add my file “script.sh” from my HDFS directory. log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Kindly if you can assist. ... View more

Hi,   I am running a spark job but it fails.  I have added the oozie-libpath as well. Still it is unable to locate my jar. Could you please help. @sonups3 wrote: Thanks for the reply, i've solved this issue.   In the workflow setting of Hue Oozie, under variables add oozie-libpath and value is the full path of the jar. In the jars option just specify the name of the jar. It works   ... View more