Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie Spark Action - HDP 2.5.3.0-37

oozie Spark Action - HDP 2.5.3.0-37

New Contributor

Hi All,

I am trying to run a spark action in oozie and getting Null Pointer exception. Below is my workflow.xml and exception details...

workflow.xml

	<workflow-app xmlns="uri:oozie:workflow:0.5" name="Spark_Test">
<global>
<job-tracker>${job_tracker}</job-tracker>
<name-node>${name_node}</name-node>
</global>
<credentials>
<credential name="hiveCredentials" type="hive2">
<property>
<name>hive2.jdbc.url</name>
<value>${hive_beeline_server}</value>
</property>
<property>
<name>hive2.server.principal</name>
<value>${hive_kerberos_principal}</value>
</property>
</credential>
</credentials>
<start to="SparkTest" />
<action name="SparkTest" cred="hiveCredentials">
<spark xmlns="uri:oozie:spark-action:0.1">
<job-tracker>${job_tracker}</job-tracker>
<name-node>${name_node}</name-node>
<master>yarn-client</master>
<name>Spark Hive Example</name>
<class>com.fbr.genjson.exec.GenExecJson</class>
<jar>${jarPath}/fedebomrpt_genjson.jar</jar>
<spark-opts>--jars /usr/hdp/current/spark-client/lib/datanucleus-api-jdo-3.2.6.jar,/usr/hdp/current/spark-client/lib/datanucleus-rdbms-3.2.9.jar,/usr/hdp/current/spark-client/lib/datanucleus-core-3.2.10.jar --files /etc/hive/conf/hive-site.xml --conf spark.sql.hive.convertMetastoreOrc=false --driver-memory 2g --executor-memory 16g --executor-cores 4 --conf spark.ui.port=5051 --queue fbr</spark-opts>
<arg>${arg1}</arg>
<arg>${arg2}</arg>
</spark>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Spark Java PatentCitation failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>

Exception:

SERVER[xxx.hpc.xx.com] USER[prxtcbrd] GROUP[-] TOKEN[] APP[Spark_Test] JOB[0004629-170625082345353-oozie-oozi-W] ACTION[0004629-170625082345353-oozie-oozi-W@SparkTest] Error starting action [SparkTest]. ErrorType [ERROR], ErrorCode [NullPointerException], Message [NullPointerException: null]
org.apache.oozie.action.ActionExecutorException: NullPointerException: null
	at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1202)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1373)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:232)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
	at org.apache.oozie.command.XCommand.call(XCommand.java:287)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
	at org.apache.oozie.action.hadoop.SparkActionExecutor.setupActionConf(SparkActionExecutor.java:85)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1091)
	... 11 more

Don't have an account?
Coming from Hortonworks? Activate your account here