oozie spark action returns Intercepting System.exit(101) on most basic spark application

[#1]

 Executing a real simple spark job, just print one line.

The jar file is generated by sbt. It works fine with spark-submit command line.

When executing from oozie (BY hue's UI, only specified the jar file, no arguments, no 3-party jars):

  Always return System.exit(101). No matter the spark.core dependency is included or not.

Does some knows the meaning of error 101? Can't find it in the source code in SparkMain or SparkSubmit.

Stdout:

....

                 --conf
                    spark.yarn.security.tokens.hive.enabled=false
                    --conf
                    spark.yarn.security.tokens.hbase.enabled=false
                    --verbose
                    /user/mrt1/test/oozie-test.jar

=================================================================

>>> Invoking Spark class now >>>

Intercepting System.exit(101)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], exit code [101]

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher, uploading action data to HDFS sequence file: hdfs://.....:8020/user/mrt1/oozie-oozi/0000142-160913091926881-oozie-oozi-W/spark-57f6--spark/action...

Oozie Launcher ends

[#2 · Accepted solution]

Hello,

kind of figured out a few things here ... there was a mental block on local vs hdfs paths that the error messages didn't really give clues. the property file defines the oozie.wf.application.path which is the base directory for the supplied jars, the jar path was preceeded by ${nodeName} which is an absolute path.

you have to logically put together the Configuration (property) file with the Definition (workflow.xml) to make sure every thing is being addressed properly.

Also digging in the spark code yielded the System.exit(101)  error ... classNotFound, which was an indicator that an invalid path was being used in our configuration.

hope this helps,

Craig