Support Questions

gimp077 · ‎03-31-2017

I set up my workflow put it into hdfs as well and I try to run the conf directory the properties file with this syntax.

I am really not sure why it is not working if I have a typo in my workflow.xml or job.properties or if I need to modify some config setting. Thanks

Error message

Here is the link to the error message, https://ibb.co/dkHnJv

tseader · ‎04-01-2017

Pease show your job.properties file and your oozie submit command please.

A few base questions:

Are you defining your oozie.wf.application.path properly in your job.properties?
Do you have a correct reference to your namenode that perhaps you are referencing in your workflow

You might also try looking at the Oozie logs, which may give you more details on the error.

gimp077 · ‎04-02-2017

Properties file 

# Environment settings
queueName      = default
kerberos_realm = A
jobTracker     = B:8032
nameNode       = hdfs://nameservice1
hive2_server   = C
hive2_port     = 10000
impala_server  = D:21000
edge_server    = E
jobTracker     = yarnrm
# Project specific paths
projectPath                  = /user/${user.name}/oozie/mediaroom-logs
keyTabLocation               = /user/${user.name}/keytabs

# job path
oozie.wf.application.path = ${projectPath}/BXDB/wf

# Project specific jars and other libraries
oozie.libpath                = ${projectPath}/lib,${projectPath}/util

# Standard useful properties
oozie.use.system.libpath     = true
oozie.wf.rerun.failnodes     = true

# Keytab specifics
keyTabName                   = A.keytab
keyTabUsername               = A
focusNodeLoginIng            = A
focusNodeLogin               = A

# Email notification list
emailList = B

xml file 

<workflow-app xmlns="uri:oozie:workflow:0.4" name="bxdb">

	<global>
		<job-tracker>${jobTracker}</job-tracker>
		<name-node>${nameNode}</name-node>
	</global>

	<credentials>
		<credential name="hive2_credentials" type="hive2">
			<property>
				<name>hive2.jdbc.url</name>
				<value>jdbc:hive2://${hive2_server}:${hive2_port}/default</value>
			</property>
			<property>
				<name>hive2.server.principal</name>
				<value>hive/${hive2_server}@${kerberos_realm}</value>
			</property>
		</credential>
	</credentials>

	<start to="sshFileTransfer"/>

	<action name="sshFileTransfer">
		<ssh xmlns="uri:oozie:ssh-action:0.1">
			<host>${focusNodeLoginIng}</host>
			<!-- Change the name of the script -->
			<command>/A/B/EsdToHDFS.sh</command>
			<args>A</args>
			<args> B</args>
			<args> C</args>
			<capture-output />
		</ssh>
		<ok to="process-bxdb"/>
		<error to="sendEmailDQ_SRC"/>
	</action>



	<!-- Move from landing zone on HDFS to processing -->


	<!-- Emit whether data is complete or partial, together with timestamp -->


	<!-- Spark job to process the snapshots and cdr data -->
	<action name="process-bxdb">
		<spark xmlns="uri:oozie:spark-action:0.2">
			<master>yarn</master>
			<mode>cluster</mode>
			<name>Process BXDB</name>
			<class>IngestBXDB</class>
			<jar>bxdb_sproc_cataloguereport-1.0-SNAPSHOT.jar</jar>
			<spark-opts>--num-executors 8  --executor-cores 2 --executor-memory 4G --driver-memory 4g --driver-cores 2</spark-opts>
			<arg>${nameNode}/user/hive/warehouse/belltv_lnd.db/bxdb_sproc_cataloguereport</arg>
			<arg>Hello</arg>
			<arg>World</arg>
		</spark>
		<ok to="impala-refresh-iis"/>
		<error to="sendEmailDQ_SRC"/>
	</action>











	<!-- Impala invalidate/refresh metadata -->
	<action name="impala-refresh-iis">
		<shell xmlns="uri:oozie:shell-action:0.3">
			<exec>impala-command.sh</exec>
			<argument>${keyTabName}</argument>
			<argument>${keyTabUsername}</argument>
			<argument>${impala_server}</argument>
			<argument>refresh belltv_expl.bxdb_sproc_cataloguereport</argument>
			<file>${nameNode}/${keyTabLocation}/${keyTabName}</file>
		</shell>
		<ok to="end"/>
		<error to="fail"/>
	</action>



	

	<action name="sendEmailDQ_SRC">
		<email xmlns="uri:oozie:email-action:0.1">
			<to>${emailList}</to>
			<subject>Error in the workflow please verify</subject>
			<body>BXDB project returned an error please verify</body>
		</email>
		<ok to="fail"/>
		<error to="fail"/>
	</action>


	<kill name="fail">
		 <message>"BXDB  ingestion failure"</message>
	</kill>

	<end name='end'/>
</workflow-app>

command to run

oozie job -abc.properties -run

tseader · ‎04-03-2017

Try addig some arguments into your Oozie run command like so:

$ oozie job -oozie http://localhost:11000/oozie -config job.properties -run

If those changes don't work for you might try the following:

Put your job.properties out in HDFS in the same directory as your workflow, then use Hue FileBrowser to execute the workflow and see if that works. To do that, just checkmark the workflow.xml and a button will appear for you to take action like a submit.
Reduce your workflow down to a simple email, then test... add the SSH, then test... keep adding and testing along the way. If things fail at the first and most simple test (email action), then we've eliminated the other actions as being the culprit, and likely quite a few of your job.properties variables too.

Cloudera Community

Support Questions

Trying to run an oozie job receiving a null pointer exception