Let me explain the issue, it's turning me crazy...

I have a spark2 program that I want to submit from an oozie workflow.

Because spark2 by default cannot be submitted directly from oozie, I have created an sh with the spark2-submit sentence to execute the spark2 job.

If I run this sh from the console it works perfect. However, when I run it from the oozie workflow, there is no way to make it work, but the worse is I cannot see a clear error on logs returned by the execution.

These are the files I am using:

thintest.sh:

spark2-submit --master yarn --class main hdfs:///home/cloudera/thintest/thintest-assembly-0.1.jar

job.properties

oozie.use.system.libpath=True
dryrun=False
nameNode=hdfs://quickstart.cloudera:8020
jobTracker=quickstart.cloudera:8032
security_enabled=False
hiveXml=/etc/hive/conf/hive-site.xml
appName=thintest
appPath=${nameNode}/home/cloudera/thintest
oozie.wf.application.path=${appPath}/workflow.xml
shellPath=${appPath}/thintest.sh

workflow.xml

<?xml version="1.0" encoding="UTF-8"?>
<workflow-app name="thintest" xmlns="uri:oozie:workflow:0.5">

<start to="shell"/>

  <action name="shell">
    <shell xmlns="uri:oozie:shell-action:0.1">

      <job-tracker>${jobTracker}</job-tracker>
      <name-node>${nameNode}</name-node>

      <exec>${shellPath}</exec>
      <env-var>HADOOP_USER_NAME=${wf:user()}</env-var>
      <file>${shellPath}#thintest.sh</file> 

    </shell>

    <ok to="end"/>
    <error to="kill"/>

  </action>

  <kill name="kill">
    <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
  </kill>

  <end name="end"/>
</workflow-app>

Any help will be very appreciated.